Systems and methods for assessing drug efficacy

ABSTRACT

Provided is a computer-implemented method, including inputting to a trained machine learning classifier genomic information of a non-training subject that includes features from a tumor sample, wherein the trained machine learning classifier trained on features of tumor samples obtained from training subjects and their a responsiveness to checkpoint inhibition treatment and the machine-learning classifier is trained to predict responsiveness to the treatment, and generating a checkpoint inhibition responsiveness classification predictive of the subject&#39;s responding to the checkpoint inhibition with the trained machine-learning classifier, and reporting the checkpoint inhibition responsiveness classification using a graphical user interface. Also provided are a computer system for performing the method and a machine learning classifier trained by the method.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationNo. 62/593,802, filed Dec. 1, 2017, the entire contents of which arehereby incorporated herein.

BACKGROUND OF THE INVENTION

Detecting abnormal or cancerous cells in the body is an important taskof the immune system. One mechanism involved is the immune checkpoint.For example, programmed cell death protein 1 (PD-1) and cytotoxicT-lymphocyte-associated protein 4 (CTLA-4) checkpoints on T-cellsnegatively regulate immune function and prevent overreaction (i.e.,promote immune system self-recognition). However, this mechanism can beexploited by tumor cells to escape immune attacks. Immunotherapies suchas PD-1 inhibition (e.g., anti-PD1 antibodies) and CTLA-4 (e.g., CTLA-4antibodies) block check point activity, hence facilitating T-cellidentification of disease or tumor cells as such.

However, although immune checkpoint therapies can be effective,responsiveness in all cancer patients is not guaranteed. Compared totraditional cancer therapies, immune checkpoint therapies have shownimprovement of long term survival of patients with various cancers.However, only a subset of cancer patients responds to currently approvedcheckpoint inhibitor drugs including anti-CTLA-4 antibody (e.g.,ipilimumab), and treatments targeting the PD-1 checkpoint pathway suchas anti-PD-1 antibody (e.g., nivolumab) or anti-programmed death-ligand1 (anti-PD-L1) antibody (e.g., atezolizumab). Therefore, it would beadvantageous to be able to select patients who would respond toparticular checkpoint therapies, and to predict which checkpoint targetpermits the best outcome in a given patient.

Many different genomic and cellular features may contribute to theeffectiveness of immunotherapy for a given individual. A higher tumormutation burden (TMB), for example, may positively affect response rateby increasing antigens presented on tumor cells, resulting in increasedrecognition by T-cells when PD-1 is blocked. CD4/CD8/CD19-expressingleukocyte tumor infiltration correlates with better clinical outcomesince such cells help immunological attack of tumor cells and subsequentantigen release. Myeloid derived suppression cells and regulatory Tcells (Tregs) sequestrate T-cell availability and correlate with worsesurvival in various patients. Since they are detectable and derivablefeatures from next generation sequencing (NGS) data that interplay witheach other, it is important to build a machine learning application thatinterrogate their relationship to immunotherapy responses and produces aprediction of responsiveness to therapy, such as checkpoint inhibition,or other cancer treatment, incorporating the context of many featuresacting in concert in a given individual based on responsiveness ofothers in view of their individual multifactorial contexts.

Furthermore, given the potentially large number of interacting genomic,cellular, and other features that may interact to determine whether agiven individual will respond positively to checkpoint inhibition, animproved method of reporting responsiveness prediction is needed. Forexample, many different features may combinatorially interact inpredicting responsiveness. When a machine learning method is applied toassess whether a patient may be more or less responsive to a givencheckpoint inhibition, some features may be determined to have greateror lesser importance than others, different features may differ fromvarious degrees from a level that suggests each may influenceresponsiveness, and different factors may signal a patient's greater orlesser responsiveness to different checkpoint inhibition treatments, indifferent individuals. Thus, a contextual report of a given patient'sresponsiveness including identification of features with significance inpredicting responsiveness and the directionality in their effect of theprediction is required. However, given limited space for presentation ofall such potential aspects of a prediction report, current reportingmethods are insufficient. Thus, a new method for reporting a multitudeof elements of a responsiveness prediction related is desired.

The present disclosure is directed to overcoming these and otherdeficiencies in the art.

SUMMARY OF THE INVENTION

In an aspect, disclosed is a computer-implemented method, includinginputting to a trained machine learning classifier genomic informationof a non-training subject, the genomic information of the non-trainingsubject comprising features from a tumor profile obtained from thenon-training subject, wherein the trained machine learning classifiertrained on genomic information of a plurality of training subjects and aresponsiveness of each of the plurality of training subjects to atreatment including a checkpoint inhibition, the genomic information ofthe plurality of training subjects comprising features of tumor samplesobtained from each of a plurality of training subjects, wherein themachine-learning classifier trained to predict responsiveness to thetreatment; generating a checkpoint inhibition responsivenessclassification for the non-training subject using the trainedmachine-learning classifier, the checkpoint inhibition responsivenessclassification predictive of the non-training subject responding to thecheckpoint inhibition; and reporting the checkpoint inhibitionresponsiveness classification of the non-training subject using agraphical user interface. In an example, at least some of the featuresfrom a tumor profile obtained from the non-training subject or at leastsome of the features from a tumor profile obtained from one or more ofthe training subjects are selected from the following group of features:total mutational burden consisting of all mutations, total mutationalburden consisting of non-synonymous mutations, beta 2 microglobulin(B2M) expression, proteasome subunit beta 10 (PSMB10) expression,antigen peptide transmitter 1 (TAP1) expression, antigen peptidetransporter 2 (TAP2) expression, human leukocyte antigen A (HLA-A)expression, major histocompatibility complex class I B (HLA-B)expression, major histocompatibility complex class I C (HLA-C)expression, major histocompatibility complex class II DQ alpha 1(HLA-DQA1) expression, HLA class II histocompatibility antigen DRB1 betachain (HLA-DRB1) expression, HLA class I histocompatibility antigenalpha chain E (HLA-E) expression, natural killer cell granule protein 7(NKG7) expression, chemokine like receptor 1 (CMKLR1) expression, tumorinfiltration by cells expressing cluster of differentiation 8 (CD8),tumor infiltration by cells expressing cluster of differentiation 4(CD4), tumor infiltration by cells expressing cluster of differentiation19 (CD19), granzyme A (GZMA) expression, perforin-1 (PRF1) expression,cytotoxic T-lymphocyte-associated protein 4 (CTLA4) expression,programmed cell death protein 1 (PD1) expression, programmeddeath-ligand 1 (PDL1) expression, programmed cell death 1 ligand 2(PDL2) expression, lymphocyte-activation gene 3 (LAG3) expression, Tcell immunoreceptor with Ig and ITIM domains (TIGIT) expression, clusterof differentiation 276 (CD276) expression, chemokine (C-C motif) ligand5 (CCL5), CD27 expression, chemokine (C-X-C motif) ligand 9 (CXCL9)expression, C-X-C motif chemokine receptor 6 (CXCR6), indoleamine2,3-dioxygenase (IDO) expression, signal transducer and activator oftranscription 1 (STAT1) expression, 3-fucosyl-N-acetyl-lactosamine(CD15) expression, interleukin-2 receptor alpha chain (CD25) expression,siglec-3 (CD33), cluster of differentiation 39 (CD39) expression,cluster of differentiation (CD118) expression, forkhead box P3 (FOXP3)expression, and any combination of two or more of the foregoing.

In another example, at least some of the training features or at leastsome of the non-training features include gene sets. In a furtherexample, the gene sets were selected using single sample gene setenrichment analysis. In yet another example, the machine learningclassifier is random forest. In a still further example, at least 50,000trees are used in training the machine learning classifier. In yet afurther example, the checkpoint inhibition responsiveness classificationincludes a prediction score and one or more feature identifiers, and theone or more feature identifiers are selected from the group consistingof a feature valence, a feature importance, and a feature weight.

In another example, the graphical user interface reports featureidentifiers as aspects of an annulus sector, wherein an angle of theannulus sector reports the feature importance, an outer radius of theannulus sector reports the feature weight, and a color of the annulussector reports the feature valence. In a further example, featureimportance of a feature includes a Gini index decrease of the feature.In still another example, the graphical user interface reports anidentifier of a feature if and only if the feature importance of thefeature is above a threshold. In yet another example, the featureimportance of the feature is not above the threshold if the square ofthe feature importance of the feature is not above 0.1. In still afurther example, each of the annulus sectors includes an inner arc andthe inner arcs of the annulus sectors are arranged to form a circle.

Another example further includes inputting to the trained machinelearning classifier a responsiveness of the non-training subject to thetreatment and further training the machine learning classifier, whereinfurther training includes training the trained machine learningclassifier on features of tumor samples obtained from the non-trainingsubject and a responsiveness of the non-training subject to thetreatment. Yet another example further includes selecting a treatmentbased upon the generated checkpoint inhibition responsivenessclassification.

In another aspect, disclosed is a computer system, including one or moremicroprocessors and one or more memories for storing a trained machinelearning classifier and genomic information of a non-training subject,wherein the trained machine learning classifier trained on genomicinformation of a plurality of training subjects and a responsiveness ofeach of the plurality of training subjects to a treatment comprising acheckpoint inhibition, the genomic information of the plurality oftraining subjects comprising features of tumor profiles obtained fromeach of the plurality of training subjects, and the machine-learningclassifier trained to predict responsiveness to the treatment, and thegenomic information of the non-training subject comprising features froma tumor profile obtained from the non-training subject, and the one ormore memories storing instructions that, when executed by the one ormore microprocessors, cause the computer system to generate a checkpointinhibition responsiveness classification for the non-training subjectusing the trained machine-learning classifier and report the checkpointinhibition responsiveness classification of the non-training subjectusing a graphical user interface, the checkpoint inhibitionresponsiveness classification predictive of the non-training subjectresponding to the checkpoint inhibition.

In an example, at least some of the features from a tumor profileobtained from the non-training subject or at least some of the featuresfrom a tumor profile obtained from one or more of the training subjectsare selected from the following group: total mutational burdenconsisting of all mutations, total mutational burden consisting ofnonsynonymous mutations, beta 2 microglobulin (B2M) expression,proteasome subunit beta 10 (PSMB10) expression, antigen peptidetransmitter 1 (TAP1) expression, antigen peptide transporter 2 (TAP2)expression, human leukocyte antigen A (HLA-A) expression, majorhistocompatibility complex class I B (HLA-B) expression, majorhistocompatibility complex class I C (HLA-C) expression, majorhistocompatibility complex class II DQ alpha 1 (HLA-DQA1) expression,HLA class II histocompatibility antigen DRB1 beta chain (HLA-DRB1)expression, HLA class I histocompatibility antigen alpha chain E (HLA-E)expression, natural killer cell granule protein 7 (NKG7) expression,chemokine like receptor 1 (CMKLR1) expression, tumor infiltration bycells expressing cluster of differentiation 8 (CD8), tumor infiltrationby cells expressing cluster of differentiation 4 (CD4), tumorinfiltration by cells expressing cluster of differentiation 19 (CD19),granzyme A (GZMA) expression, perforin-1 (PRF1) expression, cytotoxicT-lymphocyte-associated protein 4 (CTLA4) expression, programmed celldeath protein 1 (PD1) expression, programmed death-ligand 1 (PDL1)expression, programmed cell death 1 ligand 2 (PDL2) expression,lymphocyte-activation gene 3 (LAG3) expression, T cell immunoreceptorwith Ig and ITIM domains (TIGIT) expression, cluster of differentiation276 (CD276) expression, chemokine (C-C motif) ligand 5 (CCL5), CD27expression, chemokine (C-X-C motif) ligand 9 (CXCL9) expression, C-X-Cmotif chemokine receptor 6 (CXCR6), indoleamine 2,3-dioxygenase (IDO)expression, signal transducer and activator of transcription 1 (STAT1)expression, 3-fucosyl-N-acetyl-lactosamine (CD15) expression,interleukin-2 receptor alpha chain (CD25) expression, siglec-3 (CD33),cluster of differentiation 39 (CD39) expression, cluster ofdifferentiation (CD118) expression, forkhead box P3 (FOXP3) expression,and any combination of two or more of the foregoing.

In another example, at least some of the training features or at leastsome of the non-training features include gene sets. In yet anotherexample, the gene sets were selected using single sample gene setenrichment analysis. In still another example, the machine learningclassifier is random forest. In a further example, at least 50,000 treesare used in training the machine learning classifier. In yet a furtherexample, the checkpoint inhibition responsiveness classificationcomprises a prediction score and one or more feature identifiers, andthe one or more feature identifiers are selected from the groupconsisting of a feature valence, a feature importance, and a featureweight. the instructions, when executed by the one or moremicroprocessors, cause the graphical user interface to report featureidentifiers as aspects of an annulus sector, wherein an angle of theannulus sector reports the feature importance, an outer radius of theannulus sector reports the feature weight, and a color of the annulussector reports the feature valence.

In another example, feature importance of a feature comprises a Giniindex decrease of the feature. In yet another example, the instructions,when executed by the one or more microprocessors, cause the graphicaluser interface to report an identifier of a feature if and only if thefeature importance of the feature is above a threshold. In yet a furtherexample, the feature importance of the feature is not above thethreshold if the square of the feature importance of the feature is notabove 0.1. In still another example, the instructions, when executed bythe one or more microprocessors, cause the graphical user interface toreport an inner arc of each of the annulus sectors and a circlecomprising the inner arcs of the annulus sectors. In still a furtherexample, the instructions, when executed by the one or moremicroprocessors, cause the computer system to further train the machinelearning classifier, wherein further training includes training thetrained machine learning classifier on features of tumor samplesobtained from the non-training subject and a responsiveness of thenon-training subject to the treatment.

In yet another aspect, disclosed is a machine learning-based classifierfor classification of immune checkpoint responsiveness, the machinelearning-based classifier including a machine learning-based classifier,running on numerous processors, trained to predict responsiveness of anon-training subject to an immune checkpoint inhibition treatment,wherein the machine learning-based classifier trained by inputting, tothe machine-learning based classifier, genomic information of aplurality of training subjects and a responsiveness of each of theplurality of training subjects to the treatment, the genomic informationof the plurality of training subjects comprising features of tumorprofiles obtained from each of the plurality of training subjects, aninput processor that inputs features of tumor samples obtained from thenon-training subject into the machine learning-based classifier, whereinthe machine-learning classifier is configured to generate a checkpointinhibition responsiveness classification for the non-training subject,the checkpoint inhibition responsiveness classification predictive ofthe subject responding to checkpoint inhibition treatment; and an outputprocessor that reports checkpoint inhibition responsivenessclassification. In an example, the checkpoint inhibition responsivenessclassification includes a prediction score and a plurality ofidentifiers.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features, aspects, and advantages of the presentdisclosure will become better understood when the following detaileddescription is read with reference to the accompanying drawings,wherein:

FIG. 1 is a web diagram showing options for performing a method inaccordance with aspects of the present disclosure.

FIG. 2 shows some non-limiting examples of features that may be relevantin training a classifier and predicting a patient's responsiveness totreatment in accordance with aspects of the present disclosure.

FIG. 3 is a web diagram showing an example of how a method of training aclassifier may be performed in accordance with aspects of the presentdisclosure.

FIG. 4 is a web diagram showing an example of how a method of using atrained classifier to predict a subject's responsiveness to treatmentmay be performed in accordance with aspects of the present disclosure.

FIG. 5 is an example of a method of reporting a subject's responsivenessto a treatment as predicted by a trained machine learning-basedclassifier in accordance with aspects of the present disclosure.

FIG. 6 is an example of a method of reporting a subject's responsivenessto a treatment as predicted by a trained machine learning-basedclassifier in accordance with aspects of the present disclosure.

FIG. 7 is an example of a method of reporting and comparing a subject'sresponsiveness to different treatments as predicted by a trained machinelearning-based classifier in accordance with aspects of the presentdisclosure.

FIG. 8 is an example of a method of reporting and comparing a subject'sresponsiveness to different treatments as predicted by a trained machinelearning-based classifier in accordance with aspects of the presentdisclosure.

FIG. 9 is an example of a method of reporting and comparing a subject'sresponsiveness treatments as predicted by a trained machinelearning-based classifier in when gene sets are not or are included asfeatures in accordance with aspects of the present disclosure.

FIG. 10 is an example of a method of reporting a subject'sresponsiveness to a treatment as predicted by a trained machinelearning-based classifier in accordance with aspects of the presentdisclosure.

FIGS. 11A-11D are comparative graphs and figures demonstrating thatusing 38 features generates a superior classifier performance comparedto using single factors.

FIGS. 12A-12D are graphs and figures comparing performance of machinelearning classifiers using 38 features without gene sets or 44 featureswith gene sets.

DETAILED DESCRIPTION OF THE INVENTION

This disclosure relates to a machine learning method for obtaining aprediction of whether an individual may respond to treatment with acheckpoint inhibitor or other cancer treatment. Different individualsmay or may not respond to a given treatment. Responsivity to a treatmentmay depend not simply on the presence or absence of a given feature orquantitative amount of a given feature in isolation. Rather, a number offeatures may combine differentially between individuals to render someindividuals more likely to responds to a given treatment and otherindividuals less so. Standard methods of predicting patientresponsiveness on the basis of a single feature, or even severalfeatures, do not accurately predict responsiveness under circumstanceswhere a large number of factors, which may very independently of eachother, act in concert in this way.

A machine learning method presents a novel solution to this shortcomingof prevailing diagnostic prediction methods. In supervised machinelearning, for example, large data sets representing many individuals'numerous features, paired with each individual's known responsiveness toa given treatment, may be loaded onto computer memory storage. Thestorage medium of the computer contains instructions that instruct thecomputer processor to process individuals' feature information andresponsiveness to identify patterns in feature information that signal ahigh or low likelihood of an individual being responsive to the giventreatment. An advantage of using a machine learning method for such ananalysis is that it permits identification of patterns in features andtheir relevance to predicting responsiveness to treatment that are notpossible without the high data storage and retrieval capabilities ofcomputer systems and their ability to process high amounts ofinformation. A computer-implemented machine learning system may processtens or hundreds of features or more of tens or hundreds of subjects ormore to identify patterns of features and their aggregate correlationsto the individuals' responsiveness. Patterns so identified may beotherwise undetectable, requiring as they do the processing of largeamounts of data in complex sets of information.

To determine features of an individual, a tissue sample of theindividual, such as an individual with cancer, may be obtained andcharacteristics of such tissue determined. In some examples, obtainedtissue may be a sample of cells taken from a tumor. In other examples,obtained tissue may be non-tumor tissue obtained from the individual.

Here, checkpoint inhibition refers to treatments that block a process bywhich tumor cells activate self-recognition pathways in the immunesystem and thereby prevent immune cell attack and cytolysis of tumorcells. Activation of such pathways by tumor cells is thought tocontribute to the refractoriness of some patients to immuno-oncologytreatments, such as where immune cells are engineered to recognize andtarget tumor cells. Examples of checkpoint inhibition include the CTLA4pathway. CTLA4 is a protein receptor expressed on regulatory T cells. Itmay also be expressed on conventional T cells following activationthereof, as is often seen in cancer. When CTLA4 binds CD80 or CD87,proteins expressed on the surface of antigen-presenting cells,immunosuppression results. In healthy cells this mechanism promotesself-recognition and prevents immunological attack of self-cells.However, in cancer, upregulation of this pathway helps tumor cells evadedetection and attack by the immune system. Examples of checkpointinhibitor treatments that inhibit this pathway (such as theanti-CTLA4-antibody ipilimumab) may promote the ability of the immunesystem to target and destroy tumor cells, such as when paired with otherimmuno-oncological treatments that stimulate anti-tumorimmunoresponsiveness.

Similarly, another example of checkpoint inhibition is the PD-1 pathway.Binding of PD-1 on T cells by the PD1-L1 receptor expressed onself-cells causes an immunosuppressive response. As with the CTLA4pathway, this pathway is also utilized by tumor cells to evade detectionand attack by the immune system. Examples of checkpoint inhibitortreatments that inhibit this pathway (such as the anti-PD-1 antibodiespembrolizumab, nivolumab, and cemiplimab and the anti-PD-L1 antibodiesatezolizumab, avelumab, and durvalumab) may promote the ability of theimmune system to target and destroy tumor cells, such as when pairedwith other immuno-oncological treatments that stimulate anti-tumorimmunoresponsiveness.

As used herein, the term checkpoint inhibitor or checkpoint inhibitontreatment or the like includes these treatments as well as othertreatments that inhibit checkpoint inhibitor pathways, includingtreatment with other antibodies or pharmaceutical compounds thatfunction by preventing CTLA4 or PD-1 interaction with their cognateligands or receptors or activation of their downstream signalingsequelae or cellular functions.

Many features may be relevant in producing a prediction of whetherindividuals' responsiveness to a given treatment. Examples may includegenetic sequence information contained in the genome of cells taken froman individual, genetic sequence information expressed in RNA transcribedfrom the genome of an individual's cell, amounts of expression oftranscripts of genomic sequences which may be reflected in amounts of acorresponding RNA transcript or protein product thereof in a sample, ortypes of cells present in a sample. In an example where a sampleincludes tumor tissue or cells from an individual, such information mayindicate characteristics of tumor cells, i.e. cells with a modifiedgenomic sequence or sequences compared to the general population ornon-tumor cells of the individual or a reference genome referred to in agenetic sequencing paradigm. Tumorigenesis may result from a change inthe nucleotide sequence or more than one change and/or of more than onesequence in genomic DNA. In some cases, for example, an accumulation ofmultiple such sequence modifications may together function to convert acell from a non-diseased cell to a tumor cell. In other cases, aninitial accumulation of one or several such modifications in a cell maypredispose the cell to accumulating further such modifications. In stillother cases, a proliferation of such modifications in a cell may signifynot that any particular modifications directly participate in or areresponsible for development of a tumor. Rather, a tumorigenic processmay result in some modifications that directly induce transformation ofa cell into a tumor cell, but may also create other modifications thatdo not.

Thus, some individuals' tumor cells may have a large number of suchmodifications to sequences of genomic DNA, whereas other individuals'may have fewer. Of those, some may result in transcripts or proteinproducts with amino acid sequences altered as a result of the genomicmodification, such as where a modification of DNA sequence in genomicDNA results in production of a protein or RNA molecule having a sequencethat differs from that which would have been produced had themodification not occurred. Such a modifications are referred to asnon-synonymous mutations. Other modifications may be of non-coding DNA,or may be modifications of coding DNA that do not alter protein aminoacid sequences. For example, modifications to intronic sequences ornon-transcribed DNA may not result in protein products whose amino acidsequence differs from that of proteins produced from a genome notcarrying the same modification. Such modifications are referred to assynonymous mutations. Thus, different tumors may contain different totalnumbers of modifications to genomic DNA, including different totalnumbers of non-synonymous mutations, different numbers of synonymousmutations, or different numbers of both (or the same total number ofgenomic mutations but different numbers of synonymous mutations anddifferent numbers of non-synonymous mutations). The number of mutationsa cell carries is referred to as its mutational burden or totalmutational burden, whereas the total number of non-synonymous mutationsit carries is referred to as its non-synonymous mutational burden andthe number of synonymous mutations it carries is referred to as itssynonymous mutational burden.

The conversion of a cell from a non-tumor cell to a tumor cell maycorrespond to an accumulation of such modifications to genomic DNA. Suchaccumulation may be an accumulation of synonymous mutations, ofnon-synonymous mutations, or of both types. Either way, total mutationalburden may increase leading up to, upon, and following conversion from anon-tumor cell to a tumor cell. Furthermore, a tumor cell's mutationalburden may influence whether a checkpoint inhibitor is likely to beeffective in stimulating an anti-tumor response by the immune system.The more mutations a tumor cell carries, the greater chance thatsuppressing checkpoint inhibition may disinhibit the immune system fromrecognizing it as a diseased cell and attacking it. In particular,non-synonymous tumor mutational burden may be positively correlated withthe ability of checkpoint inhibition to disinhibit an antitumor immuneresponse. Proteins with mutated amino acid sequences, producedconsequential to non-synonymous mutations, can be identified in cells asabnormal and presented on cell membranes as a signal of a disease stateoccurring within the cell. For example, tumors may express proteins withmutated amino acid sequences, referred to as neoantigens. Tumor cellsexpressing such neoantigens may express mutated fragments of suchneoantigens on their cell membranes.

Such neoantigen presentation may stimulate recognition by the immunesystem that the cell is diseased (e.g., a tumor cell) and promotetargeted destruction of such cells by the immune system. Acountervailing process in tumors, however, may use the checkpointpathway to evade immune detection. Thus, whether a checkpoint inhibitormay assist in enhancing immuno-oncological treatment may depend on atumor's mutational burden. A higher burden may correspond to a higherlevel of neoantigen presentation, increasing the chances of stimulatedantitumor immunogenicity when checkpoint inhibition treatment is given.A higher total mutational burden may signify greater neoantigenexpression, in that on average a higher total mutational burden maysignify a higher non-synonymous mutational burden. Furthermore, a highernon-synonymous tumor mutational burden may also signify greaterneoantigen expression and thus a higher likelihood that checkpointinhibition would be effective. It is also possible that synonymous tumormutational burden may be correlated with responsiveness to checkpointinhibition, and/or that some combination of synonymous andnon-synonymous tumor mutational burden, such as may be reflected intotal mutational burden, may correlate with responsiveness to checkpointinhibition. Thus, total mutational burden, non-synonymous tumormutational burden, synonymous tumor mutational burden, or anycombination of two or more thereof, may be predictive of checkpointresponsiveness, and may be a feature or features included in a machinelearning method as disclosed herein.

Aside from or in addition to whether a mutation is synonymous ornon-synonymous, other or additional characteristics of a mutation mayalso be present and the number or type thereof, as with whether amutation or mutations are synonymous or non-synonymous, may be relevantin predicting responsiveness to treatment. For example, some mutations,referred to as nonstop mutations, are mutations within a stop codon thatresult in production of an RNA product translation of which continuespast where in would otherwise stop due to the mutated portion of the RNAtranscript. Another form of mutation is a frame shift mutation, whichincludes an insertion (frame shift insertion) or deletion (frame shiftdeletion) of a number of contiguous nucleotides that is not divisible bythree (for example, a single nucleotide insertion or deletion), leadingto a shifting of the read sequence of codons thus resulting inrecruitment of different tRNA molecules during translation of theresulting RNA transcript and thus altered amino acid sequences oftranslated protein. Other mutations may be splice site mutations, whichoccur at or near splice sites thus modifying normal mRNA splicing andresulting in modified RNA transcripts. Or a mutation may be a missensemutation, in which a single nucleotide is changed such that a codon inwhich it contains is altered so as to recruit a different species oftRNA during translation and thus production of a protein with adifferent amino acid sequence.

Another possible mutation may be a start mutation, which is a mutationto a transcriptional start site or to a start codon, leading to changesin where transcription or translation begins, respectively. For example,a start site mutation may prevent initiation of transcription from thatstart site. Or, a mutation could create a transcriptional start sitethat was not previously present. A transcriptional start site mutationmay lead to transcription of an RNA transcript that, though of adifferent length than would otherwise have been produced, is in-framewith such transcript produced in the absence of the mutation, or thatmay be out of frame therewith. Analogous mutations to a start codon mayalso occur, leading to transcription of an RNA product from whichinitiation of translation does not occur, or where initiation oftranslation occurs from which it would not previously have beeninitiated. Such stop codon mutations may be in-frame or out of frame aswell. Or, a mutation may be a nonsense mutation, i.e., a mutation thatleads to an RNA transcript with a premature stop codon. Any of theforegoing mutations may be a single nucleotide polymorphism (SNP). Anyone or more of the foregoing types of mutations may be features withrelevance to predicting an individual's responsiveness to a giventreatment, such as with a given checkpoint inhibitor.

In another example, an amount of infiltration of a tumor by lymphocytesmay be predictive of responsiveness to a checkpoint inhibitor. Tumorscontain not only cells that have transformed from non-tumor into tumorcells but also other, non-transformed cells. Examples include cells ofthe immune system that do or may play a role in stimulating an immuneresponse against transformed cells within the tumor. Immune cells, inparticular lymphocytes, that are comingled with transformed cells in atumor are referred to a tumor infiltrating lymphocytes. Levels of tumorinfiltrating lymphocytes, and tumor infiltrating lymphocytes expressingdifferent markers that serve as identifiers of lymphocyte phenotype, maybe predictive of whether a subject from which a tumor sample was takenwill be responsive to checkpoint inhibition. Because a tumor is aheterogeneous mixture of, for example, tumor cells and tumorinfiltrating lymphocytes, it may be advantageous to distinguishlymphocyte marker expression on tumor infiltrating lymphocytes presentin the sample ant lymphocyte markers potentially expressed on othercells within the tumor sample such as transformed tumor cells.

For example, tumor infiltrating lymphocytes may express transcripts(e.g., RNA) of genes encoding cluster of differentiation 8 (CD8),cluster of differentiation 4 (CD4), or cluster of differentiation 19(CD19), each of which may serve as a marker of lymphocyte phenotype whenexpressed in a cell. Thus, levels of expression of CD8, CD4, CD19, orany combination of any two or more of the foregoing, may be determinedfrom a tumor sample. For example, amount of RNA therefore may bedetermined from the sample. Because such determination may reflect notonly expression thereof by tumor infiltrating lymphocytes but also beother cells within the tumor sample such as transformed tumor cells, itmay be advantageous to determine how much of the detected amount ofexpression thereof is attributable to expression by tumor infiltratinglymphocytes and how much is not. In order to do so, a process ofdeconvolution may be applied, whereby a level of expression by tumorinfiltrating lymphocytes may be determined as opposed to expression byother cells. Various options for performing tumor infiltratinglymphocyte deconvolution are available, including as described in, forexample, Gaujoux et al. (2013) CellMix: a comprehensive toolbox for geneexpression deconvolution. Bioinformatics 29:2211-2212, and Finotello etal. (2018), Quantifying tumor-infiltrating immune cells fromtranscriptomics data, Cancer Immunology, Immunotherapy 67:1031-1040.Deconvolution analysis may be performed in, as a non-limiting example, Rprogramming language.

In particular, by a process referred to as tumor infiltrating lymphocytedeconvolution, the level of expression of a given lymphocyte transcript(e.g., CD8, CD4, or CD19) can be used to determine what percentage oflymphocyte cells in a tumor sample are CD4-expressing or CD8 expressingor CD19 expressing. That is, more than merely signifying the amount oflymphocyte infiltration of a tumor as represented by the amount of agiven lymphocyte transcript that may be identified in a tumor, tumorinfiltrating lymphocyte deconvolution can further provide an indicationof which type of lymphocyte, identified by species of transcriptexpressed, accounts for what percentage of overall infiltration of thetumor. Total amount of lymphocyte infiltration of a tumor may berelevant in predicting whether an individual will be responsive to agiven treatment such as a checkpoint inhibitor, and the specificcontribution of lymphocyte infiltration of the tumor made by lymphocytesexpressing a given transcript (for example, but not limited to, CD4 orCD8 or CD19) may also be relevant in making such a prediction.

Various other features may be relevant in predicting whether anindividual will be responsive to checkpoint inhibition. Such featuresmay be generally classified according to processes theoretically relatedto whether a checkpoint inhibitor may or may not be effective inpromoting or enhancing an onco-immunological response. For example, somefeatures may relate to whether and to what degree tumor cells may bemore or less likely to express antigens of mutated proteins on theircell surfaces and thereby increase the chances of an anti-tumor immuneresponse. Examples of such features already discussed include types oftumor mutational burden, such as total tumor mutational burden ornon-synonymous tumor mutational burden, for example. Other examplesinclude expression levels of different proteins or transcripts of genesencoding proteins known to be involved in antigen presentation forstimulation of an immune response. Some non-limiting examples includebeta 2 microglobulin (B2M), proteasome subunit beta 10 (PSMB10), antigenpeptide transmitter 1 (TAP1), antigen peptide transporter 2 (TAP2),human leukocyte antigen A (HLA-A), major histocompatibility complexclass I B (HLA-B) expression, major histocompatibility complex class I C(HLA-C), major histocompatibility complex class II DQ alpha 1(HLA-DQA1), and HLA class II histocompatibility antigen DRB1 beta chain(HLA-DRB1). These gene products are known to play various steps in thepresentation of protein fragments, or antigens, or a cell's surface andrecognition by immune T cells.

Levels of expression of any one, or any combination of two or more, orany of these gene products in a tumor could indicate levels of antigenexpression on the surface of tumor cells. For example, expression ofsuch products may influence the degree of presentation of proteinproducts of genomic DNA bearing a non-synonymous mutation on a cell'ssurface. Where expression increases antigen presentation, likelihood ofpresenting mutated antigens that T cells are likely to recognize assignifying a diseased cell and consequently trigger an anti-tumor immuneresponse may increase the chances that a given checkpoint inhibitor maybe effective in producing a response in a subject. Thus, whereexpression level of any or combination of two or more of the foregoingis positively or negatively correlated with degree of antigenpresentation on cells within a tumor sample, such expression level maybe positively or negatively correlated, respectively, with likelihoodthat the subject from which the tumor was samples would respond to agiven checkpoint inhibitor.

Another type of feature may include level of expression of T cells or NKcells present in a tumor sample taken from a subject and which may alsobe relevant to a prediction of responsiveness to a treatment such ascheckpoint inhibition. For example, level of expression of HLA class Ihistocompatibility antigen alpha chain E (HLA-E), natural killer cellgranule protein 7 (NKG7), chemokine like receptor 1 (CMKLR1), or anycombination of two or more thereof, may also be predictive of responseto a checkpoint inhibitor. Thus, a feature may include a measure ofexpression of one or more of these products or RNA transcriptstherefore.

Another type of feature may relate to presence or levels of expressionof proteins or transcripts therefor that are related to or indicative ofenhanced cytolytic activity such as may be promoted by an anti-tumorimmune response, or which may inhibit such activity. Tumor infiltratinglymphocyte deconvolution measures discussed above may be examples ofsuch features (for example, deconvolution of tumor infiltration by cellsexpressing cluster of differentiation 8 (CD8), cluster ofdifferentiation 4 (CD4), or cluster of differentiation 19 (CD19)). Othernon-limiting examples of this category of feature may include levels ofexpression of granzyme A (GZMA) or perforin-1 (PRF1) or any combinationof two or more of the foregoing, or RNA transcripts therefor.

Still other features may be related to processes or functions incheckpoint inhibition of anti-tumor immune responsiveness. Levels ofexpression of various protein products or transcripts therefor ofplayers in checkpoint inhibition within a tumor sample from a subjectmay be relevant in predicting whether the subject will respond totreatment with a cancer therapy such as a checkpoint inhibitiontreatment. Examples of such features may include expression of, or ofRNA transcripts for, cytotoxic T-lymphocyte-associated protein 4(CTLA4), programmed cell death protein 1 (PD1), programmed death-ligand1 (PDL1), programmed cell death 1 ligand 2 (PDL2), lymphocyte-activationgene 3 (LAG3), T cell immunoreceptor with Ig and ITIM domains (TIGIT),cluster of differentiation 276 (CD276), or any two or more of theforegoing.

Other features that may be relevant in predicting whether an individualwill respond to treatment include expression of proteins, or RNAtranscripts therefor, related to interferon y activity such as productswhose expression is downstream of interferon y release and activity at areceptor therefore. Examples of this type of feature may includeexpression of, or expression of RNA transcripts for, chemokine (C-Cmotif) ligand 5 (CCL5), CD27, chemokine (C-X-C motif) ligand 9 (CXCL9),C-X-C motif chemokine receptor 6 (CXCR6), indoleamine 2,3-dioxygenase(IDO), signal transducer and activator of transcription 1 (STAT1), orany combination of two or more of the foregoing. Other indicators ofinterferon y activity may also be predictive of responsiveness totreatment such as checkpoint inhibition.

Other features that may be relevant in predicting whether an individualwill respond to treatment include expression of proteins, or RNAtranscripts therefor, related to myeloid-derived suppressor cells (MDSC)or regulatory T cells (Treg), which may confer immunosuppressive effectson anti-tumor immune responsiveness and may blunt or preventeffectiveness of immuno-oncology treatments. Examples of such featuresmay include expression from a tumor sample from a subject tumor3-fucosyl-N-acetyl-lactosamine (CD15), interleukin-2 receptor alphachain (CD25), siglec-3 (CD33), cluster of differentiation 39 (CD39),cluster of differentiation (CD118) expression, forkhead box P3 (FOXP3),or any combination of two or more of the foregoing. Tumor expressionlevels of other species of protein or corresponding RNA transcriptsindicative of presence of such cells or their activity may also berelevant to whether an individual will respond to checkpoint inhibitortreatment or other therapies for cancer.

Any one or more of any of the foregoing features may be relevant, todifferent degrees, to making a prediction as to whether an individualwill respond to treatment with a given cancer treatment, includingtreatment with a checkpoint inhibitor. Any of the features may relate toor embody genomic information regarding the subject's tumor which wassamples and tested for determination of the feature. In this case, theterm genomic is used to include not only information related to thesequence of nucleotides in genomic DNA (such as, for example, featuresrelated to mutational burden). Here, genomic information represented byfeature measures also includes measures of levels of expression ofvarious products of genome transcription or protein products producedfrom such transcripts. Thus, levels of expression of any of thedifferent protein products described above, or other protein productsinvolved in similar pathways as those specifically identified, or levelsof expression of RNA transcripts therefore, may be included in genomicinformation as it relates to predictive features as disclosed herein.Also included in genomic information related to features may be measuresof tumor infiltrating lymphocyte deconvolution features.

In addition to measures of individual features, patterns of correlatedexpression levels of features known or believed to be related to a givenpathway or function or cell type may also be features with relevance toresponsiveness to checkpoint inhibition or other cancer treatment. Forexample, of the foregoing features, groups of some sharing commonalitiesof pathways or cellular or physiological responsiveness or indicationsof cellular phenotype may be identified and a determination made basedon measurement of the individual features whether they, as a group, arecoordinately up-regulated or down-regulated or more generally expressedor otherwise present as a group in a correlatedly high or low level in asample from a given subject's tumor. In some examples, a measure of suchgeneralized measurement of grouped features may itself be entered as afeature, in addition to individual features, for training a machinelearning classifier, predicting a subject's responsiveness to checkpointinhibition or other treatment, or both. Here, such groupings of featuresto obtain an additional feature to represent the expression level, etc.,of the grouping as a whole is referred to as a gene set. Thus, a geneset may include a combinatorial measure representing a correlationalindication of presence of genomic mutation, expression levels ofparticular RNA transcripts, presence of identified cell types, etc.

For a non-limiting example, of the foregoing features, some are relatedto antigen presentation, whereby cells such as tumors express proteinfragments on their cell membrane for monitoring by the immune system. Asdescribed above, antigen presentation my increase likelihood ofstimulating an anti-tumor immune response such as with a checkpointinhibitor. Some examples of such features may include total mutationalburden, non-synonymous mutational burden, or other mutational burden(nonstop mutational burden, frame shift mutational burden (insertional,deletional, or either), splice site mutational burden, missensemutational burden, start mutational burden, (in-frame, out-of-frame, oreither), nonsense mutational burden, start codon mutational burden(including start codon SNP or other), in-frame insertion mutationalburden, in-frame deletional mutational burden, or other SNP mutationalburden, or any combination of two or more of the foregoing. Othernon-limiting examples of features pertaining to antigen presentation mayinclude beta 2 microglobulin (B2M), proteasome subunit beta 10 (PSMB10),antigen peptide transmitter 1 (TAP1), antigen peptide transporter 2(TAP2), human leukocyte antigen A (HLA-A), major histocompatibilitycomplex class I B (HLA-B) expression, major histocompatibility complexclass I C (HLA-C), major histocompatibility complex class II DQ alpha 1(HLA-DQA1), and HLA class II histocompatibility antigen DRB1 beta chain(HLA-DRB1). In addition to features related to presence or expressionlevels, etc., related to individual examples from among the foregoingfeatures, an additional feature may represent a degree to which some orall of the foregoing are coordinately up- or down-regulated, orotherwise present in high or low levels in a subject's tumor (whetherfor machine learning classifier training or prediction).

As another non-limiting example, some features are related to level ofexpression of T cells or NK cells present in a tumor sample taken from asubject and which may also be relevant to a prediction of responsivenessto a treatment such as checkpoint inhibition. For example, level ofexpression of HLA class I histocompatibility antigen alpha chain E(HLA-E), natural killer cell granule protein 7 (NKG7), chemokine likereceptor 1 (CMKLR1), or any combination of two or more thereof, may alsobe predictive of response to a checkpoint inhibitor. In addition tofeatures related to presence or expression levels, etc., related toindividual examples from among the foregoing features, an additionalfeature may represent a degree to which some or all of the foregoing arecoordinately up- or down-regulated, or otherwise present in high or lowlevels in a subject's tumor (whether for machine learning classifiertraining or prediction).

As another non-limiting example, some features are related to indicatorsof immunologically stimulated cytolysis, such as when an immune responsepromotes cell death and cell lysis such as of tumor cells, present in atumor sample taken from a subject and which may also be relevant to aprediction of responsiveness to a treatment such as checkpointinhibition. For example, deconvoluted CD8 expression, deconvoluted CD4expression, deconvoluted CD19 expression (deconvolution representingproportional contribution CD8, CD4, or CD19-expressing cells represent,respectively, relative to the number of tumor infiltrating lymphocytespresent in a tumor sample), levels of expression of granzyme A (GZMA) orperforin-1 (PRF1), or any combination of two or more of the foregoing,or RNA transcripts therefor, may also be predictive of response to acheckpoint inhibitor. In addition to features related to presence orexpression levels, etc., related to individual examples from among theforegoing features, an additional feature may represent a degree towhich some or all of the foregoing are coordinately up- ordown-regulated, or otherwise present in high or low levels in asubject's tumor (whether for machine learning classifier training orprediction).

As another non-limiting example, some features are related to cellularand molecular processes involved in checkpoint inhibition functionspresent in a tumor sample taken from a subject and which may also berelevant to a prediction of responsiveness to a treatment such ascheckpoint inhibition. Non-limiting examples of such features mayinclude expression of, or of RNA transcripts for, cytotoxicT-lymphocyte-associated protein 4 (CTLA4), programmed cell death protein1 (PD1), programmed death-ligand 1 (PDL1), programmed cell death 1ligand 2 (PDL2), lymphocyte-activation gene 3 (LAG3), T cellimmunoreceptor with Ig and ITIM domains (TIGIT), cluster ofdifferentiation 276 (CD276), or any two or more of the foregoing. Inaddition to features related to presence or expression levels, etc.,related to individual examples from among the foregoing features, anadditional feature may represent a degree to which some or all of theforegoing are coordinately up- or down-regulated, or otherwise presentin high or low levels in a subject's tumor (whether for machine learningclassifier training or prediction).

As another non-limiting example, some features are related to indicatorsor cellular and molecular pathways participating in interferon yactivity present in a tumor sample taken from a subject and which mayalso be relevant to a prediction of responsiveness to a treatment suchas checkpoint inhibition. Non-limiting examples of such features mayinclude expression of, or expression of RNA transcripts for, chemokine(C-C motif) ligand 5 (CCL5), CD27, chemokine (C-X-C motif) ligand 9(CXCL9), C-X-C motif chemokine receptor 6 (CXCR6), indoleamine2,3-dioxygenase (IDO), signal transducer and activator of transcription1 (STAT1), or any combination of two or more of the foregoing. Inaddition to features related to presence or expression levels, etc.,related to individual examples from among the foregoing features, anadditional feature may represent a degree to which some or all of theforegoing are coordinately up- or down-regulated, or otherwise presentin high or low levels in a subject's tumor (whether for machine learningclassifier training or prediction).

As another non-limiting example, some features are related to MDSC orTreg presence or activity present in a tumor sample taken from a subjectand which may also be relevant to a prediction of responsiveness to atreatment such as checkpoint inhibition. Non-limiting examples of suchfeatures may include expression from a tumor sample from a subject tumor3-fucosyl-N-acetyl-lactosamine (CD15), interleukin-2 receptor alphachain (CD25), siglec-3 (CD33), cluster of differentiation 39 (CD39),cluster of differentiation (CD118) expression, forkhead box P3 (FOXP3),or any combination of two or more of the foregoing. In addition tofeatures related to presence or expression levels, etc., related toindividual examples from among the foregoing features, an additionalfeature may represent a degree to which some or all of the foregoing arecoordinately up- or down-regulated, or otherwise present in high or lowlevels in a subject's tumor (whether for machine learning classifiertraining or prediction).

Thus, in some examples, one or more gene sets may be identified and ameasure of the coordinate or correlated degree of up- or down-regulationin a subject's tumor of features related to such gene set may beprovided as an additional feature for training a machine-learningclassifier or predicting a subject's responsiveness to checkpointinhibition or other treatment or both. Examples of gene sets includegene sets related to antigen presentation, signatures of T cell and NKcells, cytolysis indicators, checkpoint inhibition, interferon γ, andMSDC/Treg presence or activity. In some cases, one or more such geneset, together with one or more of any of the other individual featuresdiscussed above, may be included in training or a machine learningclassifier its use in predicting a patient's responsiveness tocheckpoint inhibition or other treatment. Various methods may beemployed in ascertaining a generalized measure of how features within agene set feature are coordinately up- or down-regulated or otherwiseexpressed or present at high or low levels coordinately or in acorrelated manner. One example may include an analysis referred to assingle sample gene set enrichment analysis (ssGSEA). ssGSEA uses anempirical cumulative distribution function to ascertain such groupedenrichment of a gene set, as described in, for example, Barbie et al.(2009), Systematic RNA interference reveals that oncogenic KRAS-drivencancers require TBK1, Nature 462:108-112. ssGSEA may be performed, as anon-limiting example, in R programming language.

For the foregoing features, some, or any combination of any two or more,may be used both for training a machine learning classifier as well asfor using a trained machine learning classifier to predict whether asubject will or how likely the subject is to respond to treatment suchas with a checkpoint inhibitor. It is not necessary that all of theforegoing features be used to train a machine learning classifier,however. A machine learning classifier may be trained by a set offeatures that includes all of the foregoing, or excludes any one or moreof the foregoing. All combinations and permutations suggested by thisoptional inclusion and exclusion are hereby incorporated in theirentirety though not necessarily explicitly recited verbatim. A skilledperson would be capable of conceptualizing subsets, combinations,sub-combinations, and permutations possible with the foregoing features.Likewise, additional features may be included as well, whether inaddition to all of the foregoing or merely combination,sub-combinations, permutations, or other mixes of fewer than all of theforegoing features. All such different examples are expressly includedin the present disclosure.

In some examples, the features of any subject used to train the machinelearning classifier are the same as the features used for any and allother subjects in training the machine learning classifier. However, inother examples, different features may be provided for different subjectused to train the machine learning classifier. In other words, somesubjects may have features included in their training set that are notincluded in features from the training sets of other subjects.Similarly, in some examples, to obtain from a machine learningclassifier a prediction related to a subject's responsiveness to atreatment, features obtained from a tumor sample from the subject forobtaining the prediction may be the same as features used in trainingthe classifier. That is, features from all subjects used to train themachine learning classifier may all be the same as each other's, andalso the same as the subject for whom a prediction is sought from themachine learning classifier. In other examples, there may be a mismatchbetween trained subjects' features used to train a machine learningclassifier and a subject for whom a prediction is sought from themachine learning classifier. Features from some or all subjects used totrain the machine learning classifier may include features for whichthere are not corresponding features from the subject for whom aprediction is sought from the machine learning classifier.

In some examples, a subject for whom a prediction is sought from amachine learning classifier may lack a feature that corresponds to afeature from one or some or all subjects used to train the machinelearning classifier. In other examples, the subject may have a similarfeature but not an identical feature, and the similar feature may beused in place of the absent identical feature of the subject. Forexample, a machine learning classifier may have been trained on featuresthat, for at least some training subjects, include one or more gene setfeatures, such as may be obtained using ssGSEA as described above. Forsome of the training subjects for whom gene sets were used to train themachine learning classifier, some such sets may have been obtained fromthe same underlying individual features. For example, a gene set ofantigen presentation-related gene set feature may have been obtainedfrom the same underlying features for all subjects used for training themachine learning classifier. In other examples, some anantigen-presentation related gene set for one training subject may bebased on underlying features that include some such features that werenot included in ascertaining an antigen-presentation related gene setfrom another training subject. This may also be true for other genesets. Furthermore, a gene set feature subject from whom a prediction issought from a trained machine learning classifier may be used inobtaining the prediction, and the feature value for the gene set mayhave been obtained from an underlying set of individual features fromthe subject that does not include at least one or more underlyingfeatures that had been used in obtaining the corresponding gene setfeature value for one or more of the training subjects and used intraining the machine learning classifier.

Features may be ascertained by known methods of determining geneticsequencing data or levels of protein or RNA transcript expression in abiological sample. For example, the significant amount of nucleotidesequence information that can be obtained using next generationsequencing technologies may provide both genome-related features (e.g.,total mutational burden, etc.) as well as levels of expression of, forexample, RNA transcripts, depending on the type of next generationsequencing used to obtain a given feature. Examples of appropriatemethods include whole genome sequencing, whole exome sequencing, wholetranscriptome sequencing, mRNA sequencing, gene array analysis, RNAarray analysis, protein analysis such as protein array, or other relatedmethods for ascertaining presence or levels or amounts of features usedto train and/or obtain a prediction from a machine learning classifierin accordance with aspects of the present disclosure. In some examples,the same set of techniques may be used for obtaining features from alltraining subjects for training the machine learning classifier, and froma subject for whom a prediction is sought. In other examples, there maybe methodological differences between how a feature or some featureswere determined for different training subjects, and/or for how featuresused for obtaining a prediction were obtained for a subject for whom aprediction is sought.

In addition to training a machine learning classifier with features fromtraining subjects, training subjects' responsiveness to a treatment isalso loaded into a machine learning classifier. Thus, a training subjectis a subject for whom features and a responsiveness are provided totrain a machine learning classifier. Responsiveness may be a binaryclassification, such as if a training subject is classified as havingresponded to a treatment if the subject exhibited a predefined response,including an extended life span, a shrinking of a tumor, partial orcomplete remission, etc.). In other examples, responsiveness may be ascore or value based on a degree of responsiveness obtained rather thana binary assessment of whether responsiveness was or was not obtained.For example, a machine learning classifier according to the presentdisclosure may include classification and regression trees depending onthe type of prediction sought, as non-limiting examples.

A machine learning classifier may be any classifier that is suitable forcomputer-based machine learning. A non-limiting example includes arandom forest machine learning classifier. In a random forest machinelearning classifier, decision trees based on training subjects featuresand responsiveness to treatment value are created, with nodesrepresenting classification decision points and leaves representingoutcomes based on trained inputs. A random forest classifier may producemultiple trees using subsets of features and subsets of trainingsubjects to create numerous trees that are then aggregated. Suchmultiple decision trees containing subsets of inputs prevent overfittingand reduces error and bias in a prediction. In some examples, the moredecision trees created during training the more accurate a machinelearning classifier may result. In some examples, anywhere from 5,000 to500,000 decision trees may be created during training. For example,5,000, 10,000, 15,000, 20,000, 25,000, 30,000, 50,000, 75,000, 90,000,100,000, 125,000, 150,000, 175,000, 200,000, 225,000, 250,000, 275,000,300,000, 325,000, 350,000, 375,000, 400,000, 425,000, 450,000, 475,000,or 500,000 decision trees may be run and aggregated in a random forestmachine learning classifier. More or fewer trees may be run instead, asmay numbers in between these exemplary possibilities.

Numerous options are available for performing random forest training andgenerating a prediction from a random forest classifier. As anon-limiting example, R programming language may be used. Otherclassifiers may also be used in accordance with aspects of the presentdisclosure as well, including, without limitation, a neural networkclassifier, a support vector machine, a max entropy classifier, anextreme gradient boosting classifier, and a random fern classifier.

According to a method disclosed herein, features are obtained from eachof a number of training subjects, as are a responsiveness of each suchsubject. Such features and responsivenesses, or inputs, are entered intoa computer memory store, such as a hard drive, server, or other memorycomponent. Also stored on a memory storage feature of such computer areinstructions, contained in software, that instruct a one or moremicroprocessors. The instructions include instructions for using theinput from the training subjects to create a machine learningclassifier. A trained machine learning classifier is then stored in oneor more computer memories and may subsequently be run on features from asubject for whom a prediction as to responsiveness is sought.

As a non-limiting example, instructions may instruct one or moremicroprocessors to perform random forest training, creating decisiontrees from the training subject input for ascertaining whether thepresence, absence, level, etc., of various features are more or lesslikely to indicate responsiveness to a treatment, and to aggregate thenumerous decision trees into a trained random forest machine learningclassifier. A trained machine learning classifier according to thisexample, based on the aggregation of decision trees produced by the oneor more microprocessors when processing the features andresponsivenesses in accordance with the instructions, may then be storedin one or more memories. Subsequently, when a prediction as to whether anon-training subject (i.e., a subject whose feature values were not usedto train the trained machine learning classifier) will be responsive toa treatment such as with a particular checkpoint inhibitor, featuresobtained from a tumor sample of said non-training subject may be loadedinto one or more memories. One or more microprocessors may processinstructions such that the non-training subject's features are analyzedby the trained machine learning classifier, accessed from one or morememories by one or more microprocessors, and a prediction as to thesubject's responsiveness reported.

In such an example, the machine learning classifier was the trainedmachine learning classifier was trained on features of tumor samplesobtained from each of a plurality of training subjects and aresponsiveness of each of the plurality of training subjects to atreatment comprising a checkpoint inhibition, wherein themachine-learning classifier was trained to predict responsiveness to thetreatment. Furthermore, genomic information of a non-training subjectincluding non-training features from a subject tumor profile, or set offeature values, were input to the trained machine learning classifier togenerate a treatment responsiveness classification for the non-trainingsubject, such as a classification or score indicating a prediction ofhow the non-training subject would respond to the treatment. Thetreatment may be a checkpoint inhibition.

A checkpoint inhibition responsiveness for a non-training subjectgenerated by the trained machine learning classifier may be reported toa user. A report may include a classification of the subject indicating,in a binary manner, whether or not the non-trained subject is predictedto respond to the treatment. In other examples, a numerical score on ascale may indicate probability of responsiveness in addition to orrather than a binary classification of whether or not the non-trainingsubject is predicted to respond. In other examples, specific degrees ofresponsiveness may be reported. For example, a report may indicate ahigh likelihood of responsiveness, but responsiveness may be qualifiedas to duration or degree. In other examples, a report may indicate aprediction of a relatively lower likelihood of responsiveness but of agreater duration or degree if such non-training subject is predicted torespond.

A report of a prediction, as a score or a binary prediction of likely torespond as opposed to unlikely to respond, may be reported by agraphical user interface (GUI). For example, a computer or computersystem connected to a one or more memory and one or more microprocessorswherein a non-training subject's profile of feature values were enteredand analyzed by a trained machine learning classifier may further beconnected to a display device on which a prediction is reportedvisually. A GUI may take any of many forms. For example, a GUI may be atabulation of various aspects of features or a subset of features with ahigher degree of importance or weight in generating the prediction, aswell as whether each such factor indicated a higher or lesser likelihoodof the non-training subject responding to a treatment (i.e., thefeature's valence) in view of the feature's value for the non-trainingsubject, as further explained below. Or, a report may include differentshapes, shading, or color schemes for reporting such information.

A report of a prediction classification using a constellation offeatures as disclosed herein differentiates the subject matter disclosedherein from conventional methods for predicting responsiveness totreatment. Unlike conventional treatment prediction methods, herein isdisclosed a method employing a combinatorial approach where, in someexamples, a high number of features may be queried in context with eachother rather than one or merely a few in isolation in generating aprediction. Such a distinction of the herein disclosure overconventional methods is advantageous in that efforts heretofore topredict whether an individual will respond to a given treatment such asa checkpoint inhibitor have to date been of very limited accuracy andgeneralizability, based on limited numbers of features. As disclosedherein, an unconventional machine learning process for assessingcontributions of numerous factors and how each may independently and inconcert with others affect a prediction overcomes such limitations ofconventional prediction attempts.

Conventional methods for ascertaining a responsiveness prediction haverelated to identifying a feature or limited number of features that mayindicate a higher or lower likelihood of responsiveness to a treatment.Here, by contrast, disclosed is an unconventional approach wherein amachine learning classifier is used to assess relative contributions ofpotentially dozens of features or more simultaneously in context witheach other to formulate a prediction. Such a multifactorial approach,via an unconventional application of a machine learning classifier,provides significant benefits over currently available methods.

An advantageous feature of some GUI examples for reporting aresponsiveness prediction may be in the representation of multifacetedinformation pertaining to features relevant to the reported predictionin a relatively tight or small space such that a user may ascertainsignificant information in a compact manner. In particular, an advantageof certain examples of GUIs used in accordance with the presentdisclosure may be their sizing and configuration on a display of limitedor reduced size, or on a display upon which significant other quantitiesof information also need or are desired to be displayed. A GUI forreporting a prediction classification in accordance with the presentdisclosure may serve to pack a high amount of easily recognizable andcognizable facets of a prediction classification and/or featuresrelevant to the generation thereof into a limited display size orlimited proportion of a display. Such compact reporting improves userinterfacing with a computer system giving the report in that itexpedites the receipt and interpretation of a report and conservesdisplay space that may be required for other purposes or of limitedavailability such as where a display is a screen of a portableelectronic device such as a phone or other wireless communication device(e.g., computer tablet or phone or other portable wired or wirelessdevice). In an example where a report is presented by a GUI thatcondenses a high amount of characteristics and aspects of features intoa constricted space yet retains the ability to quickly convey highamounts of information that remains quickly and easily ascertainable,usability of a computer system is improved in that more display spaceremains available for additional purposes at the same time, or use onsmaller displays remains possible.

A report may include feature identifiers, or aspects or characteristicsof some or all features used in generating the responsivenessclassification. For example, a report may include an indication of afeature's valence, i.e. whether its value in a user's profile indicatedan increased or decreased likelihood that the subject would respond tothe treatment. That is, for a non-training subject's profile, a featuremay have a positive valence or a negative valence. A positive valencemay mean either the feature's value in training subjects tended to bepositively correlated with responsiveness to treatment and thenon-training subject's value for that feature was high, or that thefeature's value in training subjects tended to be negatively correlatedwith responsiveness to treatment and the non-training subject's valuefor that feature was low.

Another identifier may be an indication of a feature's importance ingenerating a prediction for a given machine learning classifier.Importance or a feature may indicate that it is more likely to drive aprediction in one direction or another relative to other features usedin training. For example, in some examples, a Gini decrease index may beascertained for one or more features during training. A Gini decreaseindex indicates importance of a feature in that it accounts for afeature's effect on a classifier's functioning relative to how muchinfluence other features have in driving a generated prediction. A Ginidecrease index may be determined with use of various software packages,such as by using R programming language. In some examples, importance'sof features may be used to determine which features' identifiers areincluded in a report separate from or in addition to a prediction score.For example, for a given report reported in the form of a GUI, the GUImay display features of only such features whose importance meets orexceeds a predetermined minimum importance threshold. For example, areport might include identifiers of only such features whose importance,represented numerically as Gini decrease index, squared exceeds 0.1.More or less stringent minimum importance thresholds may be set instead,or altered for given reports depending on how much information isdesired to be included along with a prediction score. The higher theminimum importance threshold, the fewer features' identifiers may beincluded in a report, and vice versa. For example, a minimum importancethreshold may be where the square of the numerical importance (e.g.,Gini decrease index) is above anywhere between 0.01 and 0.5. Otherminimum importance thresholds may be chosen anywhere between or outsideof tis range.

Another example of a feature identifier that may be included in a reportis a feature's weight. A feature's weight is a measure of the degree towhich that feature's value for a subject on its own would suggest thesubject would or would not respond to the treatment. For example, foreach feature, a single factor decision boundary may be determined. Asingle factor decision boundary is a value for that feature which bestdistinguishes between training subjects do and do not respond to thetreatment. For example, if all training subjects who responded totreatment had a value for a feature above a given amount whereas allnon-responding treatment subjects had values below that amount, thatamount could be the single factor decision boundary. In some otherexamples, some responding training subjects may have a feature valueabove some non-responding training subjects and other respondingtraining subject may have a feature value below those training subjects.Thus, in some examples, there may be a bright line for a feature's valuethat unequivocally distinguished between responders and non-responders,whereas for others there may be more overlap at the boundary of featurevalues between responders and non-responders. For examples of the lattertype, a single factor decision boundary may be chosen as a value whichprovides the maximum possible distinction between responders andnon-responders in the set of training subjects.

Weight of a feature is a measure or indication of how far a featurevalue for a non-training subject differs from the single featuredecision boundary for that feature. The more a value for a feature for anon-training subject differs from the single factor decision boundaryfor that feature based on the training subjects, the greater the weightthat feature may have in determining a responsiveness classificationprediction. For example, a feature may have a negative valence, meaninga non-training subject has a low value for a feature positivelycorrelated with responsiveness in training subjects, or a high value fora feature negatively correlated with responsiveness in trainingsubjects. If a non-training subject's value for that feature differssubstantially from the single factor decision boundary for that feature,the weight of that feature may be high. However, if another feature hasa higher importance, and a positive valence (i.e., high value for thenon-training subject and positive correlation with responsiveness intraining subjects, or low value in non-training subject and negativecorrelation with responsiveness in training subjects), it may have aproportionally stronger influence on the responsiveness predictionclassification even if its value differs less from the single factordecision boundary (i.e., has less weight).

Numerous possibilities are available for presenting feature identifiersas a component of a responsiveness prediction report and GUI. Severalspecific examples are presented herein in some detail. However, askilled person would appreciate that there may be many otherpossibilities for reporting valence and weight and importance offeatures in a GUI report and that the examples given here are notlimiting or each on its own in any way essential specifically.

A GUI may present tabulated features with rows and columns. Features maybe presented in, for example, rows, and different characteristics ofgiven features may be presented in multiple columns. For example,different columns may indicate the importance of a feature, its valence,the single factor decision boundary for that feature, a non-trainingsubject's value for that feature, optionally with a visual indication ofhow much the non-training subject's value for that feature differs fromthe single factor decision boundary for that feature, and whether thenon-training subject would be predicted to respond if the predictionwere based on that feature alone. A tabular GUI may include anycombination two or more of the foregoing. A tabular GUI report may alsoinclude an overall prediction score.

A GUI may also be a histogram. For example, columns may indicate a valuefor a given feature for a non-training subject, with a line alsoindicating the single factor decision boundary for that feature. Adifference between the value for the feature for the non-trainingsubject and the line indicating the single factor decision boundary isan identifier of weight for that feature. A line may also be drawnbetween the height of the column and the line indicating the singlefactor decision boundary. The length of the line between the two also isan indicator of weight. Valence may be indicated by a symbol below thecolumn. For example, a plus or minus sign may indicate a positive ofnegative valence, respectively. Other pairings could include upwardlyand downwardly pointing arrows, upwardly and downwardly orientedtriangles, etc., where one direction indicates a positive valence forthat feature and the other orientation indicates a negative valence.Valence may also be indicated by color or shading of the column, withcolumns of one color or shading pattern indicating one valence (positiveor negative) and columns of a different color or shading patternindicating the opposite valence. Color or shading of a line between avalue for a non-training subject's feature in the histogram and thesingle factor decision boundary for that feature may also indicatevalence. For example, if a feature's value is negatively correlated withresponsiveness in training subjects and the non-training subject's valuefor the feature is less that the single feature decision boundary, aline connecting the non-training subject's value in the bar of thehistogram report for that feature and the single factor decisionboundary may be a color or shading indicating positive valence. Whereas,if a feature's value is positively correlated with responsiveness intraining subjects and the non-training subject's value for the featureis less that the single feature decision boundary, a line connecting thenon-training subject's value in the bar of the histogram report for thatfeature and the single factor decision boundary may be a color orshading indicating negative valence. Note that in both instances thenon-training subject's value for the feature is less than the singlefactor decision boundary for that feature but its valence differs,depending on whether the feature was negatively (positive valence) orpositively (negative valence) correlated with responsiveness in trainingsubjects. The converse would also be true (i.e., where a feature'ssingle feature decision boundary is less than a non-training subject'svalue for that feature, it may have positive or negative valencedepending on whether the feature tended to be positively or negativelycorrelated with responsiveness in the training subjects, respectively).

Importance of a feature may be indicated by a symbol or other indicatornear, within, or below the column for that feature in the histogram. Forexample, size of a symbol below the bar for a feature may indicate itsimportance. Or importance may be color coded, with bars or associatedsymbols colored or shaded in such a way as to indicate the degree ofimportance. A key may accompany the histogram indicating a spectrum ofcolors or shades with higher importance indicated by a color or shademore similar to one end of the spectrum and lower importance indicatedby a color or shading pattern more similar to the other end of thespectrum. In some examples, a symbol, such as placed below a bar in ahistogram GUI report for a non-training subject's feature, may indicatewhether values for the feature were negatively or positively correlatedwith responsiveness in training subjects. For example, a plus sign andminus sign, upwards arrow and downwards arrow, upwardly-pointingtriangle and downwardly pointing triangle, or other pairings, maysignify positively and negatively correlated features. In such cases,relative sizes of such symbols may signify relative importance thereof.

In another example, a GUI report includes shapes that convey identifiersfor features. For example, each feature whose identifiers are includedin a GUI report may be represented by a shape whose dimensions, color,shading, or other aspects may designate different identifiers. Forexample, features may each be represented by rectangles, with widthrepresenting importance and height weight. A color or shading or outlinepattern of the rectangles may indicate valence. Or the features may berepresented by triangles, with the base representing importance and theheight weight, or vice versa. A triangle may point up for a feature withpositive valence and vice versa. Or a color of the triangle or shadingpattern or pattern of lines in which its outline is drawn may indicatevalence.

In another example, identifiers of a feature may be indicated in theshape of sector of an annulus, or annulus sector. The angle of theannulus sector may indicate importance and its outer radius its weight,or vice versa. Valence may be indicated by a color of the annulussector, or pattern in which it is shaded, or patterning of a line inwhich its outline is drawn. In other examples, a sector of a circle, orcircle sector, may represent identifiers of a feature, with angle andradius representing importance and weight or vice versa, and color orshading, etc., representing valence as for the example of an annulussector given above. Where feature identifiers are represented by annulussectors, in some examples all such annulus sectors may be drawn to havethe same inner radius, and the annulus sectors may be arranged such thattheir inner arcs together form an inner circle. Within the inner circleitself a prediction score or other overall summary or indication ofresponsiveness prediction or classification may be indicated. Color orshading of said inner circle may indicate whether the responsivenessclassification prediction predicts a non-training subject is likely torespond or not likely to respond. For example, where the prediction isthat the non-training subject is likely to respond, the inner circle mayhave one color or shading pattern or be drawn in a line of one pattern,whereas if the prediction is that the non-training subject is not likelyto respond the inner circle may have a different color or shadingpattern. In other examples, rather than an inner circle there may beanother inner shape such as a square or star or triangle or pentagon orother shape. Size of the inner shape may indicate the strength of theprediction, with a larger inner shape signifying higher confidence inthe prediction and vice versa.

A GUI report may also provide a user with the opportunity to seek moreinformation or launch additional software applications depending oninterest in particular features as reported in the GUI. For example, theGUI could be configured such that a user could hover a pointer or otherelement controllable by an input device such as a mouse over identifiersfor a feature, or by touching a touch screen. Orienting the element ortouching the display at a feature could open a drop-down menu withoptions that could be further selected. For example, a drop-down menucould display aspects of the feature specific to the non-trainingsubject such as its value, or a cohort range for the feature, whatpercentage the non-training subject's value for the feature representedrelative to the range of values present in the training data, orcompared to only training data for training subjects who responded inthe way the non-training subject was predicted to respond, or for theother training subjects, or the features single factor decisionboundary, or a feature's importance or correlation with responsivenessfor other treatments, or any combination of two or more of theforegoing. A drop-down menu may also present links to other programsaccessible by one or more microprocessors for further evaluating afeature or a non-training subjects prediction score, such as running adifferent machine learning classifier. By compacting such interactivityinto a GUI report, significant space and computational resources couldbe conserved and user interactivity with the computer systemsignificantly enhanced. For example, less display space would be neededfor simultaneous continual display of the GUI report during access ofdrop down menu options. Furthermore, time and computing resources wouldbe conserved as such interactivity would permit accessing multiplecomputer functions without switching between display screens orapplications.

In some examples, a trained machine learning classifier may be furthertrained. For example, one a non-trained subject's responsiveness to atreatment is ascertained, the machine learning classifier may beretrained on features including the training subjects feature values andresponsivenesses on which it was initially trained plus the featurevalues and responsiveness for the non-training subject for which aprediction was provided and responsiveness obtained. In other examples,a trained machine learning classifier may be retrained with additionalfeature values and responsiveness of a non-training subject for whom aprediction was not obtained from the trained machine learningclassifier.

In some examples, upon obtaining a response prediction classificationfor a subject, a decision of whether or not to apply a given checkpointinhibition treatment may be made. A high prediction score as to aparticular checkpoint treatment obtained from a machine learningclassifier trained as disclosed herein may be followed by a decision totreat the subject with the treatment for which the machine learningclassifier was trained to predict responsiveness. Or, a low score may befollowed by a decision not to apply such treatment. Someone obtaining aresponse prediction classification or a score indicating a suitably highlikelihood of responsiveness, or instructions to treat a subject as aresult of such a response prediction classification or a score havingbeen obtained, in accordance with methods, systems, or a machinelearning classifier as disclosed herein, may treat the subject with thetreatment. Included in the present disclosure is treating cancer in asubject by administering a checkpoint inhibition treatment in responseto obtaining a response prediction classification or a prediction score,generated in accordance with the present disclosure, indicating that thesubject would respond to such treatment, or administering such treatmenton the instructions of someone who obtained such a response predictionclassification or a prediction score.

EXAMPLES

The following examples are intended to illustrate particular embodimentsof the present disclosure, but are by no means intended to limit thescope thereof

FIG. 1 is a web diagram showing options for performing a method inaccordance with aspects of the present disclosure. Shown are potential,non-limiting examples of sources of feature values, such as the cancergenome atlas (TCGA), or data from clinical trials (such as trials fortreatment with anti-PD1 treatment, anti-CTLA4 treatment or othercheckpoint inhibitor treatment or other cancer treatment). Somenon-limiting examples of assays that may be used for obtaining featureinformation used to determine feature values are also indicted, such asRNAseq and whole exome sequencing (WES), as two non-limiting examples.Different non-limiting examples of types of features are also indicated,as are examples of which assay or assays may provide measures relevantto ascertaining a value for such feature. Examples include HLA, geneexpression, ssGSEA, tumor mutational burden, cytolytic infiltration suchas by tumor infiltrating lymphocytes (deconvoluted), CGA, neoantigens,presence of clonal and/or subclonal mutations (i.e., mutations that arepresent in descendants of a cell that initially presented a givenmutation, and others of a cell descendant from such initially mutatedcell that subsequently acquired another mutation), etc. Features are theprocessed by a machine learning (ML) model whereby a machine learningclassifier is trained and a non-training subject's feature values areinput to the trained machine learning classifier, whereupon a patientresponse is obtained for labeling the non-training patent as likely orunlikely to respond to the treatment.

FIG. 2 shows some non-limiting examples of features that may be relevantin training a classifier and predicting a patient's responsiveness totreatment in accordance with aspects of the present disclosure. As wouldbe understood by skilled artisans, the features identified in FIG. 2 arenon-limiting examples. They are also not all required. Others than thoseshown in FIG. 2 may be employed whereas some shown may be omitted inperforming a method in accordance with aspects of the presentdisclosure. Features here are shown as grouped according to function,cellular or molecular pathway or response, etc. Examples of suchgroupings include antigen presentation, T cell or NK cell signatures,signatures of immunologically mediated cytolysis, checkpoint pathwayparticipants, interferon y pathway participants, and MDSC/Tregsignatures. Other groupings representing other functions or cellular ormolecular processes, etc., may be included in addition to or instead ofany of those shown here as non-limiting examples.

FIG. 3 is a web diagram showing an example of how a method of training aclassifier may be performed in accordance with aspects of the presentdisclosure. As would be understood by skilled artisans, the featuresidentified in FIG. 3 are non-limiting examples. They are also not allrequired. Others than those shown in FIG. 3 may be employed whereas someshown may be omitted in performing a method in accordance with aspectsof the present disclosure. Features here are shown as grouped accordingto function, cellular or molecular pathway or response, etc. Examples ofsuch groupings include antigen presentation, T cell or NK cellsignatures, signatures of immunologically mediated cytolysis, checkpointpathway participants, interferon γ pathway participants, and MDSC/Tregsignatures. For training, as indicated by FIG. 3, features from trainingsubjects, such as those shown, are inputted to a machine learningclassifier such as a random forest classifier as are labelscorresponding to how the trained subjects responded to a given treatmentsuch as a checkpoint inhibition treatment. In this manner, theclassifier is trained. As disclosed, other machine learning classifiersmay also be used.

FIG. 4 is an expansion of FIG. 3, showing a web diagram showing anexample of how a method of using a trained classifier to predict asubject's responsiveness to treatment may be performed in accordancewith aspects of the present disclosure. A machine learning classifier(in this non-limiting example, a random forest classifier) having beentrained on features (non-limiting examples of which are shown here forexample), receives further input obtained from a non-training subject.In particular, feature values from the non-training subject are input tothe machine learning classifier. The trained machine learning classifierthen generates a response prediction classification which may include ascore indicating likelihood of responsiveness (here indicated asimmuno-scores) and/or identifiers of features, in reporting aprediction.

Features and responsiveness to PD-1 inhibition were obtained from Hugoet al. (2016) Genomic and Transcriptomic Features of Response toAnti-PD-1 Therapy in Metastatic Melanoma. Cell. 2016; 165(1):35-44(doi:10.1016/j.cell.2016.02.065). In that study, whole exome sequencingdata and RNAseq data were obtained from tumor samples from 26 patientswith melanoma were obtained pre- and -post treatment with an inhibitorof PD-1 checkpoint pathway (anti-PD-1 antibody treatment (nivolumab) oranti-PD-L1 antibody treatment (pembrolizumab). Raw data were publiclyavailable and accessed for the examples herein. Transcriptome dataobtained via RNAseq (including expression levels of transcripts in thesamples) was available online from the National Center for BiotechnologyInformation (NCBI) Gene Expression Omnibus at accession number GSE78220(https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE78220).Whole-exome sequencing data obtained by NGS methods was available onlinefrom the NCBI Sequence Read Archive (https://www.ncbi.nlm.nih.gov/sra)under accession numbers SRA: SRP067938 and SRA: SRP090294.Responsiveness of the patients for whom such data were available werealso obtained from the results of the published studies. Data wereselected from these sources to create features for training subjects andresponsivenesses for training subjects to train a machine learningclassifier to predict responsiveness to anti-PD1 checkpoint pathwayinhibitors. Data for features for obtaining a prediction from a trainedmachine learning classifier were also obtained from these sources.

Features and responsiveness to CTLA-4 inhibition were obtained from VanAllen et al. (2015) Genomic correlates of response to CTLA-4 blockade inmetastatic melanoma. Science 350:207-211 (doi: 10.1126/science.aad0095).In that study, whole exome sequencing data and RNAseq data were obtainedfrom tumor samples from 30 patients with melanoma were obtained pre- and-post treatment with an inhibitor of CTLA-4 (ipilimumab). Raw data werepublicly available and accessed for the examples herein. Whole exomesequencing data and transcriptome data obtained via NGS methods wereavailable online from the NCBI database of Genotypes and Phenotypes(dbGaP), accession number phs000452.v2.p1(https://www.ncbi.nlm.nih.gov/gap/?term=phs000452.v2.p1). Responsivenessof the patients for whom such data were available were also obtainedfrom the results of the published studies. Data were selected from thesesources to create features for training subjects and responsivenessesfor training subjects to train a machine learning classifier to predictresponsiveness to anti-CTLA4 checkpoint pathway inhibitors. Data forfeatures for obtaining a prediction from a trained machine learningclassifier were also obtained from these sources.

From data obtained from both studies, data was pre-processed to createfeatures input to machine learning classifiers for training. Randomforest machine learning classifiers having received such input were thentrained on training data. Features from some subjects was then used togenerate a prediction from one or both trained classifiers and aresponsiveness prediction classification was reported via a GUI,including a prediction score and identifiers of features. FIG. 1 asdescribed above presents an overview of an example of some methods usedin these examples. FIG. 2 shows 38 features that were selected forinputting to a machine learning classifier. Features were selected basedon expectations that they may relate to a prediction of whether asubject may or may not respond to checkpoint inhibition based on currentunderstanding of the roles of such features. allMut refers to tumormutational burden all mutations and nonSynMut refers to tumor mutationalburden attributable to nonsynonymous mutations. cd8_dec, cd4_dec, andcd19_dec refer to tumor infiltrating lymphocyte deconvolution measuresfor the respects CD8, CD4, and CD19. The remaining features identifiedin FIG. 2 were obtained from RNAseq data and include measures ofrelative expression levels. For some examples, ssGSEA was also obtainedfor sets of features (antigen presentation, T cell/NK cell signature,immunological cytolytic signature, checkpoint pathway, interferon γ, andMDSC/Treg signatures). Some machine learning classifiers were trained ononly the original 38 features and others were trained on the 38 originalfeatures plus the gene sets obtained by ssGSEA analysis resulting in atotal of 44 features for such examples.

Subjects' data per feature value were normalized by rank ordering theirvalues as a percentage each value per feature relative to the range ofvalues for all subject's values for that feature in the set.Responsiveness for the training in these examples was binary, with asubject labeled as either being responsive (including both partialresponsivenesses and full responsiveness or long-term benefit fromtreatment as reported in the respective underlying study) or notresponsive (including subjects reported as having disease progression inresponse to treatment and no benefit of treatment reported in theunderlying studies).

Features were inputted to a computer system including one or more memorystores and one or more microprocessors. The one or more memory storescontained instructions which, when run by the one or moremicroprocessors, trained a random forest machine learning classifierusing R programming language. Subjects features were stores on the oneor more memory stores and analyzed by the one or more microprocessorsaccording to the instructions. In these examples, 50,000 trees wereused. After training, a subject's features were the input to the trainedmachine learning classifier, as stored on the one or more memory storesof the computer system, and a response prediction classification scoregenerated and reported by a GUI on a computer display. An example oftraining is shown in FIG. 3 and an example of generating a prediction isshown in FIG. 4.

Final prediction scores were generated by the probability ofclassification, scaled to a 0-10 score. As an example, by way ofexplanation, a 0.75 probability of classified in “has response” to agiven immunotherapy was translated to a responsiveness prediction scoreof 7.5. Furthermore, for each feature, a single factor decision boundarywas determined at the value that maximize the classification accuracy.The correlation direction of whether a feature was positively ornegatively correlated with classification as responsive was determinedby the spearman correlation between the feature and response. Theforegoing analyses were carried out using full samples withoutseparation of training and non-training subjects. Performance analysiswere carried out separately. For 3-fold cross-validation and area underthe curve (AUC) plotting, R programming language package “caret” and“cvAUC” were used with default function and parameter.

FIG. 5 shows an example of a report of a GUI 500 reporting responseclassification prediction score and feature identifiers. For theseexamples, identifiers of a feature were presented only if a square ofthe feature's importance (importance being determined by the feature'sGini index decrease) was greater the 0.1. FIG. 5 shows a GUI report fora subject whose data was obtained for an anti-PD1 trial run on a machinelearning classifier trained to predict responsiveness to anti-PD1treatment (e.g., anti-PD1 or anti-PD1-L1 antibody). The 15 features withimportances that exceeded the minimum importance threshold are shown inthe FEATURES column 510. The features' importances are shown in the IMP.column 520. The group the feature is associated with as per FIG. 2 isshown in the column GROUP 530. A feature's positive or negativecorrelation with responsiveness is shown in the CORR column 540. In thisexample, triangles are used to indicate the direction of correlation,with an upwardly pointing triangle indicating positive correlation and adownwardly pointing triangle indicating negative correlation between afeature value and responsiveness. A feature's single feature decisionboundary is shown in column 1FDB 550. In this example, the single factordecision boundary is identified numerically as the percentage within therange of subject values for that sample above and below which providedthe highest obtainable distinction between responders and non-responders(i.e., a value above or below that percentage was less accurate overallin distinguishing between a responsive and a non-responsive subject).Single factor decision boundary is also indicated by shading from leftto right within each feature's cell in the 1FDB column, representing thepercentage indicated by the single factor decision boundary (i.e., a lowpercentage has less shading and a higher percentage has more shading,proportional to the value of the boundary). The subject's value for eachfeature is shown in the INPUT column 560 (in this case PT5 INPUT,identifying the patient for whom the prediction is here reported aspatient number 5, or PT5). The number in PT5 INPUT column 560 indicatesthe subject's value for a given feature as well as shading indicatingthe percentage rank of that subject's feature value relative to therange of values for training subjects.

Whether the subject would have been predicted to respond to treatmentbased solely on the value of a given feature is reported in the column1F.PRED 570. Thus, for positively correlated features, if the value inPT5 Input 560 exceeds the value in 1FDB 550, 1F.PRED 570 indicates YES(meaning that feature would predict that the subject would respond totreatment). For positively correlated features, if the value in PT5Input 560 is below the value in 1FDB 550, 1F.PRED 570 indicates NO(meaning that feature would predict that the subject would not respondto treatment). For negatively correlated features, if the value in PT5Input 560 exceeds the value in 1FDB 550, 1F.PRED 570 indicates NO(meaning that feature would predict that the subject would not respondto treatment). And for negatively correlated features, if the value inPT5 Input 560 is below the value in 1FDB 550, 1F.PRED 570 indicates YES(meaning that feature would predict that the subject would respond totreatment). In this example, cells in 1F.PRED 570 may also be colorcoded in accordance with whether the feature alone would predictresponsiveness. YES cells may be colored green, for example (R), whereasNO cells may be colored red (R). The bottom row FULL MODEL 580 reportsthe response classification score, in this case 5.5. A cutoff may bedetermined above or below which a treatment may be predicted to beeffective or not effective. For example, a score below 5.0 may be takenas a prediction that the treatment would not work for this patient and ascore above 5.0 may be taken as an indication that the treatment wouldwork for this patient. In this case, with a score above 5 being taken asan indication that the treatment would work for this patient, it can beseen that the value of training and using a machine learning classifieras disclosed herein provides a significantly improved basis forprediction than would basis prediction on only one of the features whoseindicators are presented in the GUI report, in that some features alonepredicted non-responsiveness (including the feature with the highestimportance) but the machine learning classifier overall predicts thatthe patient would respond.

In this example, a response prediction score is reported by the GUI, asare indicators for each feature included in the GUI report, includingweight (by offering a comparison between PT5 INPUT 560 and 1FDB 550),importance 520, and valence 1F.PRED. In some examples, a user could havethe option of accessing drop down menus from different aspects of theGUI such as by touching a portion of a touchscreen display correspondingto a portion of the GUI report or moving a graphical element such as acursor with a device such as a mouse over a feature or relatedindicators or scores thereof to access additional information or selectfrom additional analysis that could be run via different programminginstructions on the one or more microprocessors stored on the one ormore memory stores.

Another GUI report 610 for this subject is presented in FIG. 6. MultipleGUI reports are combined into a single report in the example presentedin FIG. 6. The upper portion of FIG. 6 presents a ring of annulussectors 610, each corresponding to a feature whose importance exceededthe minimum importance threshold as explained for FIG. 5. The feature towhich each annulus sector corresponds is also indicated in writing. Forexample, the annulus sector corresponding to feature HLA.B is indicatedby 630. Each annulus sector has an angle, an outer radius, and an innerradius, and an inner arc. In this example, the angle corresponds to thefeature's importance. In this example, the angles of the features areproportional to one another to permit direct visual comparison. Also inthis example, the difference between the outer radius and the innerradius corresponds to its weight (i.e., subject's feature value'sdifference from the single factor decision boundary). The valence of afeature in this example is also reported by the style of line whichoutlines the feature's annulus sector. A feature whose annulus sectorhas a positive valence for a subject (such as for all tumor mutationalburden 640) is outlined with a solid line while a feature whose annulussector has a negative valence for a subject (such as for HLA.B 630) isoutlined with a dotted line.

In this example, the inner arcs are arranged so as to form an innercircle. Also in this example, within the inner circle a responsivenessprediction classification score for this patient is reported, in thiscase 5.5. Also in this example, the overall prediction is indicated by asolid line forming the inner circle, meaning the subject's responseprediction classification score exceeds the predetermined leveldistinguishing between a prediction of responsiveness andnon-responsiveness. In other examples, a dotted line could form the oran inner circle if the response prediction classification score wasbelow such predetermined score threshold. In other examples, color ordifference shading patters within an annulus sector and/or inner circle,and/or differential coloring of outlines of an annulus sector or innercircle, may indicate valence.

In this example, the upper portion 610 of the GUI report presented inFIG. 6 600 includes a report of a response prediction classificationscore and features' importances, valences, and weights are indicated.Another example of such a GUI report 1010 is presented in FIG. 10.Annulus sectors for features are presented individually rather than withtheir inner arcs forming a circle. Examples of an outer arc 1002, innerarc 1001, and difference between an outer and inner radius 1003 areshown for the annulus sector for CD15 and an angle for an annular sector1004 is shown below the annulus sector for HLA.B 1030 for illustrativepurposes. Features' outer radii (or difference between inner radii andouter radii) report weight, angles report importance, and pattern ofline outlining an annulus sector report whether a feature has a positive(solid line, e.g. the annulus sector for all tumor mutational burden,all tmb 1040) or negative (dotted line, e.g. the annulus sector forCD15) valence. Not shown but also optionally included in such a GUIreport 1010 is a response prediction classification score. Rather thandifferent patterned outlines, different colors or patterned shadingcould be used to indicate a feature's valence. Shapes other than annulussectors could also be used. For example, features could be reported asrectangles, with importance and weight represented by width and height,for example, or triangles with base width reflecting importance, heightrepresenting weight, and orientation representing valence. A skilledperson would appreciate that numerous possibilities could be adapted forpurposes of reporting multiple indicators of multiple features of areport GUI in accordance with aspects of the present disclosure.

Returning to the GUI report 600 depicted in FIG. 6, below the upperportion of the report that includes annulus sectors 610 is a histogramportion of a GUI report 620. A GUI report can have both such portions oronly one, or neither. The histogram 620 shows a column for each featurewhose value for the reported on subject exceeds the minimum importancethreshold set for that feature. The scale on the left 650 indicates thepercentage rank of the subject's value for each feature. Each feature'scolumn represents the subject's feature value for that feature as apercentage of the range of values for the training subjects values. Anexample is indicated for the subject value for feature CD15 660. Alsopresented for each column as the single feature decision boundary forthat feature, as a horizontal line in this case. An example is indicatedfor the single feature decision boundary for feature CD15 670. Trianglesbelow each column indicate whether the feature is positively (upwardlypointing triangle) or negatively (downwardly pointing triangle)correlated with responsiveness. An example is indicated for CD15 680.Triangles are also proportioned to reflect relative importance of eachfeature, with larger triangles representing higher importance andsmaller triangles representing lower importance. A line between asubject's feature value and the single factor decision boundary for thatfeature indicates weight of that feature. An example of a report ofweight for a feature is indicated for feature CD15 690. Valence for afeature is indicated by whether the weight line is solid (positivevalence) or dotted (negative valence). As would be appreciated by askilled person, each of these particular examples for reportingdifferent indications of different features could be omitted, orsubstituted with different graphical representations. Colors and shadingcould represent valence and/or correlations for features, arrows orother directional shapes could represent valence or correlations,importances could be represented by a scaled color coding scheme, etc.

In some examples, gene sets were used to train a machine learningclassifier and generate a prediction from a trained machine learningclassifier. An example of gene sets, generated by ssGSEA, are shown inFIG. 7. Six sets are shown, grouped with individual features used todetermine the gene set. Examples include antigen processing pathway,i.e. related to antigen presentation (710), T cell and NK cellsignatures 720, cytolytic signatures 730, checkpoint pathway 740,interferon gamma 750, and MSDC/Treg signatures 760. Correlations andimportances for each feature, including cell sets, when used to train aPD1 and a CTLA4 machine learning classifier are indicated at 770 and780, respectively. In some instances, highlighted in cells outlined withdashed boxes, a gene set provided either a higher magnitude correlationor a higher importance than any individual feature upon which the geneset value was determined using ssGSEA, indicating the value of includinggene sets as features. Usefulness of including ssGSEA is also shown inFIG. 9. FIG. 9 shows two GUI reports on a responsiveness prediction forthe same subject using two different trained machine learningclassifiers. The prediction on the left 910 was obtained by generating aprediction without using ssGSEA gene sets as features during training orprediction. The prediction on the right 920 was obtained by generating aprediction using ssGSEA gene sets as features during training andprediction. When ssGSEA-derived gene sets were included, fewer featuresexceeded the minimum threshold boundary (11 versus 15), including somegene sets (which by definition were not included in the predictionobtained without gene sets 910), without sacrificing overall prediction(e.g., in both cases the prediction exceeded the 5.0 score set as aminimum response prediction classification score triggeringclassification of a subject as a responder).

FIG. 8 shows two GUI reports obtained from the same patient for whom aprediction was generated using two different machine learningclassifiers, one trained to predict responsiveness to anti-CTLA4treatment 810 and the other trained to predict responsiveness toanti-PD1 820, both using the features depicted in FIG. 2. The anti-CTLA4machine learning classifier 810 produced a response predictionclassification score of 3.8 predicting that the subject would notrespond to anti-CTLA4 treatment (using a response predictionclassification score threshold of 5.0). In this case 810, the valence ofthe responsiveness prediction classification score (non-responsiveness)is indicated with dotted lines around an inner circle in which theresponse prediction classification score is depicted 815. Accuracy ofsuch prediction is corroborated by the response classification of thispatient from the source clinical study (Van Allen et al.) as “diseaseprogression,” indicating that the subject was non-responsive toanti-CTLA4 treatment. However, the anti-PD1 machine learning classifier820 generated a response prediction classification score of 6.7predicting that the subject would respond to anti-PD1 (e.g., anti-PD1 oranti-PD-L1 antibody) treatment (again using a response predictionclassification score threshold of 5.0). In this case 820, the responseprediction classification score (responsiveness) is indicated with asolid line around an inner circle in which the response predictionclassification score is depicted 817.

Different response prediction classification scores (and correspondingdifferences in valence, weight, and importance of various features)depending on which machine learning classifier is used reflects thepower of methods as disclosed herein. For example, indicators reportedfor HLA.A 830 indicate that it has a negative valence for predictingresponsiveness of this patient to anti-CTLA4 treatment but a positivevalence for predicting this subject's responsiveness to anti-PD1treatment. Furthermore, values of features for nonsynonymous tumormutation burden and all tumor mutational burden did not surpass theminimum importance threshold in the anti-CTLA4 machine learningclassifier but did for the anti-PD1 machine learning classifier 840.

As with the example depicted in FIG. 5, the exemplary GUI reportsdepicted in FIGS. 6, 8, 9, and 10 could include user interactivity.Thus, for some examples, a user could have the option of accessing dropdown menus from different aspects of the GUI such as by touching aportion of a touchscreen display corresponding to a portion of the GUIreport or moving a graphical element such as a cursor with a device suchas a mouse over a feature (e.g., annulus sector or column in a histogramor other indicators) to access additional information or select fromadditional analysis that could be run via different programminginstructions on the one or more microprocessors stored on the one ormore memory stores.

FIGS. 11A-11D demonstrate that using all of the 38 features depicted inFIG. 2 generates a classifier performance than using single factors.This was true for machine learning classifiers whether trained topredict responsiveness to anti-PD1 or anti-CTLA4 treatment. FIG. 11Ashows that using all 38 features for training and testing a machinelearning classifier to predict responsiveness to, for example, anti-PD1treatment, yielded an AUC of the Receiver Operator Characteristic(auROC) of false- vs true-positives of 1.00 when the machine learningclassifier is over-trained without cross-validation (CV), reaching 1.00auROC, vs. 0.64 auROC for averaged single factors as shown in FIG. 11B.FIGS. 11C and 11D further show that using the 38 features depicted inFIG. 2 is more accurate than all three of the top single factorfeatures, HLA-B, nonSyn tmb and all tmb.

FIGS. 12A-12D are graphs showing the effects using gene sets obtainedvia ssGSEA had on classifier performance. Performance of ssGSEA iscomparable to using the 38 features depicted in FIG. 2. FIGS. 12A, 12B,and 12C report 3-fold cross validation auROC for using the 38 featuresdepicted in FIG. 2 for predicting responsiveness to anti-PD1 (FIG. 12A)or anti-CTLA4 (FIG. 12B) treatment, or using the 38 features plus the 6gene sets depicted in FIG. 7 for predicting responsiveness to anti-PD1treatment (FIG. 12C). Although performance was reduced from 0.69 to 0.64for anti-PD1 treatment prediction when the 6 gene sets attained viassGSEA were included (comparing FIG. 12A to FIG. 12C), the number offeatures with importance that exceeded the minimum importance thresholdwas reduced from 15 to 11 (as discussed above; see e.g. FIG. 8). FIG.12D shows comparable t-test results for false and true positive resultsby response prediction classification score for machine learningclassifiers (without cross-validation). In some examples, including genesets as features therefore may improve overall robustness of aclassifier, help avoid over-training, and enables a cleanerinterpretation of important features and indicators therefor.

The data used in these examples were from subjects enrolled in a studyto test effectiveness of checkpoint inhibitors anti-CTLA4 antibody,anti-PD1 antibody, and anti-PD1-L1 antibodies in patients with melanoma.However, as would be understood by skilled persons based on the rolecheckpoint inhibits are known to play in blunting effectiveness ofimmuno-oncology treatment of other cancers, and the role of featuressuch as those included here in checkpoint pathway function, methods,systems, and classifiers as disclosed herein would be equally useful andeffective in generating predictions of subject responsiveness to thesecheckpoint inhibitors in other cancers as well, including breast cancer,cancers of the digestive system, liver cancer, bladder cancer, lymphoma,leukemia, cancers of bone tissue, cancers of the nervous system, lungcancers, pancreatic cancers, or others. Furthermore, as would also beunderstood by skilled persons, responsiveness to checkpoint inhibitorsin addition to those specifically used in the foregoing examplesdisclosed here may also be predicted using methods, systems, andclassifiers as disclosed herein, which serve merely as non-limitingexamples of the applicability thereof.

The pitfall of this performance analysis is the low sample size. Evenwith dozens of features, over-training is unavoidable. Meanwhile, foldcross-validation results are highly unstable, showing discretized auROCpoints, instead of a continuous curve. However, within the limitation wehave, full model clearly out-performs single factors.

Although preferred embodiments have been depicted and described indetail herein, it will be apparent to those skilled in the relevant artthat various modifications, additions, substitutions, and the like canbe made without departing from the spirit of the present disclosure andthese are therefore considered to be within the scope of the presentdisclosure as defined in the claims that follow.

It should be appreciated that all combinations of the foregoing conceptsand additional concepts discussed in greater detail herein (providedsuch concepts are not mutually inconsistent) are contemplated as beingpart of the inventive subject matter disclosed herein. In particular,all combinations of claimed subject matter appearing at the end of thisdisclosure are contemplated as being part of the inventive subjectmatter disclosed herein.

What is claimed is:
 1. A computer-implemented method, comprising:inputting to a trained machine learning classifier genomic informationof a non-training subject, the genomic information of the non-trainingsubject comprising features from a tumor profile obtained from thenon-training subject, wherein the trained machine learning classifiertrained on genomic information of a plurality of training subjects and aresponsiveness of each of the plurality of training subjects to atreatment comprising a checkpoint inhibition, the genomic information ofthe plurality of training subjects comprising features of tumor profilesobtained from each of the plurality of training subjects, wherein themachine-learning classifier trained to predict responsiveness to thetreatment; generating a checkpoint inhibition responsivenessclassification for the non-training subject using the trainedmachine-learning classifier, the checkpoint inhibition responsivenessclassification predictive of the non-training subject responding to thecheckpoint inhibition; and reporting the checkpoint inhibitionresponsiveness classification of the non-training subject using agraphical user interface.
 2. The method of claim 1, wherein at leastsome of the features from a tumor profile obtained from the non-trainingsubject or at least some of the features from a tumor profile obtainedfrom one or more of the training subjects are selected from the groupconsisting of total mutational burden consisting of all mutations, totalmutational burden consisting of nonsynonymous mutations, beta 2microglobulin (B2M) expression, proteasome subunit beta 10 (PSMB10)expression, antigen peptide transmitter 1 (TAP1) expression, antigenpeptide transporter 2 (TAP2) expression, human leukocyte antigen A(HLA-A) expression, major histocompatibility complex class I B (HLA-B)expression, major histocompatibility complex class I C (HLA-C)expression, major histocompatibility complex class II DQ alpha 1(HLA-DQA1) expression, HLA class II histocompatibility antigen DRB1 betachain (HLA-DRB1) expression, HLA class I histocompatibility antigenalpha chain E (HLA-E) expression, natural killer cell granule protein 7(NKG7) expression, chemokine like receptor 1 (CMKLR1) expression, tumorinfiltration by cells expressing cluster of differentiation 8 (CD8),tumor infiltration by cells expressing cluster of differentiation 4(CD4), tumor infiltration by cells expressing cluster of differentiation19 (CD19), granzyme A (GZMA) expression, perforin-1 (PRF1) expression,cytotoxic T-lymphocyte-associated protein 4 (CTLA4) expression,programmed cell death protein 1 (PD1) expression, programmeddeath-ligand 1 (PDL1) expression, programmed cell death 1 ligand 2(PDL2) expression, lymphocyte-activation gene 3 (LAG3) expression, Tcell immunoreceptor with Ig and ITIM domains (TIGIT) expression, clusterof differentiation 276 (CD276) expression, chemokine (C-C motif) ligand5 (CCL5), CD27 expression, chemokine (C-X-C motif) ligand 9 (CXCL9)expression, C-X-C motif chemokine receptor 6 (CXCR6), indoleamine2,3-dioxygenase (IDO) expression, signal transducer and activator oftranscription 1 (STAT1) expression, 3-fucosyl-N-acetyl-lactosamine(CD15) expression, interleukin-2 receptor alpha chain (CD25) expression,siglec-3 (CD33), cluster of differentiation 39 (CD39) expression,cluster of differentiation (CD118) expression, forkhead box P3 (FOXP3)expression, and any combination of two or more of the foregoing.
 3. Themethod of claim 1, wherein at least some of the training features or atleast some of the non-training features comprise gene sets.
 4. Themethod of claim 3 wherein the gene sets were selected using singlesample gene set enrichment analysis.
 5. The method of claim 1, whereinthe machine learning classifier is random forest.
 6. The method of claim5, wherein at least 50,000 trees are used in training the machinelearning classifier.
 7. The method of claim 1, wherein the checkpointinhibition responsiveness classification comprises a prediction scoreand one or more feature identifiers, and the one or more featureidentifiers are selected from the group consisting of a feature valence,a feature importance, and a feature weight.
 8. The method of claim 7,wherein the graphical user interface reports feature identifiers asaspects of an annulus sector, wherein an angle of the annulus sectorreports the feature importance, an outer radius of the annulus sectorreports the feature weight, and a color of the annulus sector reportsthe feature valence.
 9. The method of claim 8 wherein feature importanceof a feature comprises a Gini index decrease of the feature.
 10. Themethod of claim 9 wherein the graphical user interface reports anidentifier of a feature if and only if the feature importance of thefeature is above a threshold.
 11. The method of claim 10 wherein thefeature importance of the feature is not above the threshold if thesquare of the feature importance of the feature is not above 0.1. 12.The method of claim 10, wherein each of the annulus sectors comprises aninner arc and the inner arcs of the annulus sectors are arranged to forma circle.
 13. The method of claim 1, further comprising inputting to thetrained machine learning classifier a responsiveness of the non-trainingsubject to the treatment and further training the machine learningclassifier, wherein further training comprises training the trainedmachine learning classifier on features of tumor samples obtained fromthe non-training subject and a responsiveness of the non-trainingsubject to the treatment.
 14. A computer system, comprising: one or moremicroprocessors, one or more memories for storing a trained machinelearning classifier and genomic information of a non-training subject,wherein the trained machine learning classifier trained on genomicinformation of a plurality of training subjects and a responsiveness ofeach of the plurality of training subjects to a treatment comprising acheckpoint inhibition, the genomic information of the plurality oftraining subjects comprising features of tumor profiles obtained fromeach of the plurality of training subjects, and the machine-learningclassifier trained to predict responsiveness to the treatment, and thegenomic information of the non-training subject comprising features froma tumor profile obtained from the non-training subject, and the one ormore memories storing instructions that, when executed by the one ormore microprocessors, cause the computer system to generate a checkpointinhibition responsiveness classification for the non-training subjectusing the trained machine-learning classifier and report the checkpointinhibition responsiveness classification of the non-training subjectusing a graphical user interface, the checkpoint inhibitionresponsiveness classification predictive of the non-training subjectresponding to the checkpoint inhibition.
 15. The system of claim 14,wherein at least some of the features from a tumor profile obtained fromthe non-training subject or at least some of the features from a tumorprofile obtained from one or more of the training subjects are selectedfrom the group consisting of total mutational burden consisting of allmutations, total mutational burden consisting of nonsynonymousmutations, beta 2 microglobulin (B2M) expression, proteasome subunitbeta 10 (PSMB10) expression, antigen peptide transmitter 1 (TAP1)expression, antigen peptide transporter 2 (TAP2) expression, humanleukocyte antigen A (HLA-A) expression, major histocompatibility complexclass I B (HLA-B) expression, major histocompatibility complex class I C(HLA-C) expression, major histocompatibility complex class II DQ alpha 1(HLA-DQA1) expression, HLA class II histocompatibility antigen DRB1 betachain (HLA-DRB1) expression, HLA class I histocompatibility antigenalpha chain E (HLA-E) expression, natural killer cell granule protein 7(NKG7) expression, chemokine like receptor 1 (CMKLR1) expression, tumorinfiltration by cells expressing cluster of differentiation 8 (CD8),tumor infiltration by cells expressing cluster of differentiation 4(CD4), tumor infiltration by cells expressing cluster of differentiation19 (CD19), granzyme A (GZMA) expression, perforin-1 (PRF1) expression,cytotoxic T-lymphocyte-associated protein 4 (CTLA4) expression,programmed cell death protein 1 (PD1) expression, programmeddeath-ligand 1 (PDL1) expression, programmed cell death 1 ligand 2(PDL2) expression, lymphocyte-activation gene 3 (LAG3) expression, Tcell immunoreceptor with Ig and ITIM domains (TIGIT) expression, clusterof differentiation 276 (CD276) expression, chemokine (C-C motif) ligand5 (CCL5), CD27 expression, chemokine (C-X-C motif) ligand 9 (CXCL9)expression, C-X-C motif chemokine receptor 6 (CXCR6), indoleamine2,3-dioxygenase (IDO) expression, signal transducer and activator oftranscription 1 (STAT1) expression, 3-fucosyl-N-acetyl-lactosamine(CD15) expression, interleukin-2 receptor alpha chain (CD25) expression,siglec-3 (CD33), cluster of differentiation 39 (CD39) expression,cluster of differentiation (CD118) expression, forkhead box P3 (FOXP3)expression, and any combination of two or more of the foregoing.
 16. Thesystem of claim 14, wherein at least some of the training features or atleast some of the non-training features comprise gene sets.
 17. Thesystem of claim 16 wherein the gene sets were selected using singlesample gene set enrichment analysis.
 18. The system of claim 14, whereinthe machine learning classifier is random forest.
 19. The system ofclaim 18, wherein at least 50,000 trees are used in training the machinelearning classifier.
 20. The system of claim 14, wherein the checkpointinhibition responsiveness classification comprises a prediction scoreand one or more feature identifiers, and the one or more featureidentifiers are selected from the group consisting of a feature valence,a feature importance, and a feature weight.
 21. The method of claim 20,wherein the instructions, when executed by the one or moremicroprocessors, cause the graphical user interface to report featureidentifiers as aspects of an annulus sector, wherein an angle of theannulus sector reports the feature importance, an outer radius of theannulus sector reports the feature weight, and a color of the annulussector reports the feature valence.
 22. The system of claim 21 whereinfeature importance of a feature comprises a Gini index decrease of thefeature.
 23. The system of claim 22 wherein the instructions, whenexecuted by the one or more microprocessors, cause the graphical userinterface to report an identifier of a feature if and only if thefeature importance of the feature is above a threshold.
 24. The systemof claim 23 wherein the feature importance of the feature is not abovethe threshold if the square of the feature importance of the feature isnot above 0.1.
 25. The system of claim 23, wherein the instructions,when executed by the one or more microprocessors, cause the graphicaluser interface to report an inner arc of each of the annulus sectors anda circle comprising the inner arcs of the annulus sectors.
 26. Thesystem of claim 14, wherein the instructions, when executed by the oneor more microprocessors, cause the computer system to further train themachine learning classifier, wherein further training comprises trainingthe trained machine learning classifier on features of tumor samplesobtained from the non-training subject and a responsiveness of thenon-training subject to the treatment.
 27. A machine learning-basedclassifier for classification of immune checkpoint responsiveness, themachine learning-based classifier comprising: a machine learning-basedclassifier, running on numerous processors, trained to predictresponsiveness of a non-training subject to an immune checkpointinhibition treatment, wherein the machine learning-based classifiertrained by inputting, to the machine-learning based classifier, genomicinformation of a plurality of training subjects and a responsiveness ofeach of the plurality of training subjects to the treatment, the genomicinformation of the plurality of training subjects comprising features oftumor profiles obtained from each of the plurality of training subjects;an input processor that inputs genomic information of the non-trainingsubject into the machine learning-based classifier, the genomicinformation of the non-training subject comprising features from a tumorprofile obtained from the non-training subject, wherein themachine-learning classifier is configured to generate a checkpointinhibition responsiveness classification for the non-training subject,the checkpoint inhibition responsiveness classification predictive ofthe subject responding to checkpoint inhibition treatment; and an outputprocessor that reports checkpoint inhibition responsivenessclassification.
 28. The machine learning-based classifier of claim 27,wherein the checkpoint inhibition responsiveness classificationcomprises a prediction score and a plurality of identifiers.